Robot control instruction compiling method and device and storage medium

Abstract

The application relates to the field of intelligent robots, and provides a robot control instruction compiling method and device and a storage medium. The method comprises the following steps: acquiring an upper-layer motion intention; generating one or more candidate semantic frames based on the upper-layer motion intention; performing standardization processing on the candidate semantic frames to generate a standardized motion intention intermediate representation, and generating a unique hash value for the standardized motion intention intermediate representation, wherein the standardization processing comprises unifying verbs, units, coordinate systems, object references and control modes; performing multidimensional verification on the standardized motion intention intermediate representation, wherein the multidimensional verification comprises mode verification, kinematics feasibility verification, control mode compatibility verification, constraint conflict verification and timing feasibility verification; after the verification is passed, constructing a motion graph based on the standardized motion intention intermediate representation, compiling the motion graph into a low-layer motion instruction, and generating a compilation certificate comprising the hash value, the compilation result and a model version.

Description

Technical Field

[0001] This application relates to the field of intelligent robots, and in particular to a method, apparatus and storage medium for compiling robot control instructions. Background Technology

[0002] With the rapid development of industrial automation and intelligent manufacturing, robots are widely used in complex work scenarios such as assembly, handling, and welding. Traditional robot control relies on professional engineers writing fixed motion programs or taught trajectories, which is not only inefficient but also difficult to adapt to frequently changing production tasks. In recent years, natural language human-computer interaction technology has gradually emerged, allowing operators to directly command robots to perform tasks through voice or text commands, thereby lowering the programming threshold and improving flexibility.

[0003] Furthermore, existing technologies lack a mechanism for compiling and verifying the executability, safety, and timing of motion intentions from AI models or advanced task planners before physical execution. These AI-generated motion intentions may contain instructions that do not conform to the robot's physical capabilities, posing significant risks to direct execution. Therefore, a runtime system capable of standardizing, verifying, and securely compiling AI motion intentions is urgently needed.

[0004] However, existing natural language robot control schemes typically parse natural language instructions directly into low-level motion commands, lacking standardized processing of command semantics and safety verification. Specifically, existing technologies suffer from the following drawbacks: First, natural language instructions often contain ambiguities, missing parameters, or non-standard expressions, and direct parsing may lead to the robot executing incorrect actions or even causing safety accidents. Second, due to the lack of intermediate representations of motion intentions and multi-dimensional verification mechanisms, it is impossible to effectively assess the kinematic feasibility, control mode compatibility, and constraint satisfaction of the instructions before execution, causing the robot to potentially operate on infeasible trajectories or exceed safety boundaries. Furthermore, existing schemes lack traceability guarantees for the source and compilation process of instructions, which is detrimental to post-event auditing and liability determination. Summary of the Invention

[0005] This application provides a robot control instruction compilation method, apparatus, and storage medium, which realizes efficient, safe, and verifiable robot control instruction compilation.

[0006] On the one hand, this application provides a method for compiling robot control instructions, the method comprising: Step S1: Obtain the upper-level motion intention, which originates from at least one of natural language commands, human-computer interaction interfaces, AI strategy models, visual language models, task planners, or script interfaces; Step S2: Based on the upper-layer motion intent, generate one or more candidate semantic frames, wherein the candidate semantic frames contain semantic information of verbs, objects, modifiers, time and force parameters; Step S3: Standardize the candidate semantic frames to generate a standardized motion intent intermediate representation and generate a unique hash value for the standardized motion intent intermediate representation. The standardization process includes unifying verbs, units, coordinate systems, object references and control modes. Step S4: Perform multi-dimensional verification on the standardized motion intention intermediate representation. The multi-dimensional verification includes pattern verification, kinematic feasibility verification, control pattern compatibility verification, constraint conflict verification, and temporal feasibility verification. Step S5: After verification, construct a motion graph based on the standardized motion intent intermediate representation, compile the motion graph into low-level motion instructions, and generate a compilation certificate containing the hash value, compilation result, and model version.

[0007] On the other hand, this application provides a robot control instruction compilation apparatus, the apparatus comprising: The acquisition module is used to acquire the upper-level motion intention, which is derived from at least one of natural language instructions, human-computer interaction interface, AI strategy model, visual language model, task planner or script interface; The first generation module is used to generate one or more candidate semantic frames based on the upper-layer motion intention. The candidate semantic frames contain semantic information of verbs, objects, modifiers, time and force parameters. The second generation module is used to standardize the candidate semantic frames, generate a standardized motion intention intermediate representation, and generate a unique hash value for the standardized motion intention intermediate representation. The standardization process includes unifying verbs, units, coordinate systems, object references, and control modes. The verification module is used to perform multi-dimensional verification on the standardized motion intention intermediate representation. The multi-dimensional verification includes pattern verification, kinematic feasibility verification, control pattern compatibility verification, constraint conflict verification, and temporal feasibility verification. The compilation module is used to construct a motion graph based on the standardized motion intent intermediate representation after verification, compile the motion graph into low-level motion instructions, and generate a compilation certificate containing the hash value, compilation result, and model version.

[0008] Thirdly, this application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the technical solution of the robot control instruction compilation method described above.

[0009] Fourthly, this application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the technical solution described above for the robot control instruction compilation method.

[0010] As can be seen from the technical solution provided in this application, on the one hand, by acquiring the upper-level motion intent and generating candidate semantic frames containing semantic information such as verbs, objects, modifiers, time, and force parameters, structured parsing of fuzzy or incomplete natural language instructions is achieved, providing an accurate semantic basis for subsequent processing. Furthermore, by standardizing the candidate semantic frames, unifying verbs, units, coordinate systems, object references, and control modes, and generating unique hash values, ambiguity caused by different expression methods is eliminated, ensuring the consistency and traceability of the intermediate representation of motion intent. On the other hand, by performing multi-dimensional verification including pattern verification, kinematic feasibility verification, control mode compatibility verification, constraint conflict verification, and temporal feasibility verification, the safety, feasibility, and compliance of motion intent can be comprehensively evaluated during the instruction compilation stage, effectively preventing the robot from executing dangerous or infeasible actions and significantly improving the robustness and safety of the system. Thirdly, after the verification is passed, a motion graph is constructed based on the standardized intermediate representation of motion intent and compiled into low-level motion instructions. At the same time, a compilation certificate containing hash values, compilation results, and model versions is generated, realizing end-to-end automated conversion from natural language to executable instructions and providing complete execution credentials, which facilitates subsequent auditing and system optimization. In summary, the technical solution of this application overcomes the ambiguity, security risks and lack of traceability of the direct parsing of natural language instructions in the prior art, and realizes efficient, safe and verifiable robot control instruction compilation. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of the robot control instruction compilation method provided in the embodiments of this application; Figure 2 This is a schematic diagram of the structure of the robot control instruction compilation device provided in the embodiments of this application; Figure 3 This is a schematic diagram of the device provided in the embodiments of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] In this specification, adjectives such as "first" and "second" are used only to distinguish one element or action from another, without necessarily requiring or implying any actual such relationship or order. Where circumstances permit, reference to an element or component or step (etc.) should not be construed as being limited to only one of the elements, components, or steps, but may be one or more of the elements, components, or steps, etc.

[0015] For ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn to actual scale.

[0016] Existing natural language robot control solutions typically parse natural language instructions directly into low-level motion commands, lacking standardized processing of command semantics and safety verification. Specifically, existing technologies suffer from the following drawbacks: First, natural language instructions often contain ambiguities, missing parameters, or non-standard expressions, and direct parsing may lead to the robot executing incorrect actions or even causing safety accidents. Second, due to the lack of intermediate representations of motion intentions and multi-dimensional verification mechanisms, it is impossible to effectively assess the kinematic feasibility, control mode compatibility, and constraint satisfaction of the instructions before execution, causing the robot to potentially operate on infeasible trajectories or exceed safety boundaries. Furthermore, existing solutions lack traceability assurance regarding the source and compilation process of instructions, hindering post-event auditing and liability determination.

[0017] To address the aforementioned problems in the prior art, this application proposes a method for compiling robot control instructions, the flowchart of which is attached. Figure 1 As shown, it mainly includes steps S1 to S5, which are detailed below: Step S1: Obtain the upper-level motion intent, wherein the upper-level motion intent originates from at least one of the following: natural language instructions, human-computer interaction interface, AI strategy model, visual language model, task planner, or script interface.

[0018] In this embodiment, upper-level motion intent refers to a high-level instruction or task description representing the robot's desired action. Its sources are diverse, including but not limited to: natural language instructions (e.g., commands input by the user via voice or text), human-computer interaction interfaces (e.g., graphical drag-and-drop, gesture recognition, touch panels, etc.), AI policy models (e.g., action sequences output by reinforcement learning-based decision models), visual language models (e.g., operational intents generated by fusing visual scenes with language descriptions), task planners (e.g., task networks output by symbolic planners or PDDL planners), and script interfaces (e.g., pre-written structured task scripts), etc. Simultaneously with acquiring upper-level motion intent, the system can also acquire information such as the robot's current state, scene state, and actuator capability descriptions, selectively acquiring these information according to actual deployment needs. Methods for acquiring upper-level motion intent include, but are not limited to: microphone array pickup and speech recognition transcription, keyboard input, API interface reception, message queue subscription, model inference output monitoring, and planner result callbacks, etc. To ensure the integrity and standardization of the input, the system can perform preliminary format verification and encoding conversion on the obtained original intent, such as removing invalid characters and unifying it to UTF-8 encoding, laying the foundation for subsequent processing.

[0019] Step S2: Based on the upper-layer motion intent, generate one or more candidate semantic frames.

[0020] In this embodiment, a candidate semantic frame is a structured semantic representation that decomposes the action intent implied in the upper-level motion intent into several semantic slots. For example, for the instruction "gently pick up the red cup and put it into the blue box," the system can generate two candidate semantic frames: the first frame has the verb "pick up," the object "red cup," the modifier "gently," an empty time (indicating immediate execution), and the force parameter "low force"; the second frame has the verb "put it into," the object "red cup," the target location "blue box," an empty modifier, and the force parameter is a default value. Each candidate semantic frame contains five core fields: verb, object, modifier, time, and force. The force parameter describes the desired force magnitude or control mode (e.g., desired stiffness under impedance control), and the time field specifies the start time, duration, or deadline of the action. The process of generating candidate semantic frames can be implemented in various ways, such as rule-based parsing, statistical sequence labeling models, or prompting engineering based on large language models. Because the upper-layer motion intent itself is ambiguous and incomplete, the system typically generates multiple candidate semantic frames to cover possible interpretations, which are then filtered and corrected through standardization and verification steps. As an embodiment of this application, generating one or more candidate semantic frames based on the upper-layer motion intent can be achieved through the following steps S2a and S2b: Step S2a: Use a large language model to extract initial semantic elements from the upper-level motion intent and generate an initial semantic frame.

[0021] Specifically, step S2a can be implemented by the system concatenating natural language instructions into prompts, inputting them into pre-trained large language models, and requiring these models to output structured JSON-formatted semantic frames. For example, the prompt could be designed as: "Please extract the verb, object, modifier, time, and force parameters from the following instructions and output JSON format: {'verb': '', 'object': '', 'modifier': '', 'time': '', 'force': ''}. Instruction: Gently pick up the red cup." Large language models, with their powerful semantic understanding capabilities, can effectively identify the core elements of instructions, and even if the instructions are not explicitly stated or contain omissions, they can infer reasonable semantics from the context. However, the output of large language models is probabilistic and may produce illusions or omissions, therefore requiring subsequent verification and correction by a rule engine.

[0022] Step S2b: Perform integrity verification and ambiguity resolution on the initial semantic frame using a predefined rule engine, supplement missing fields and correct contradictory information to generate the final candidate semantic frame.

[0023] In this embodiment, the rule engine pre-defines a series of domain knowledge rules, such as: verbs must belong to a predefined set of robot actions (e.g., move, pick, place, push, pull, etc.), objects must exist in the target list of the current scene, force parameters must be within the actuator's capabilities, and so on. Integrity verification is used to detect whether the initial semantic frame is missing necessary fields (e.g., missing verbs or objects). If missing, it is completed based on the default configuration or information from the historical context. For example, if the instruction does not explicitly specify a time, it defaults to "execute immediately"; if no force parameter is specified, it is automatically inferred based on the task type (e.g., "pick up" corresponds to low force, "push" corresponds to medium force). Ambiguity resolution is used to handle situations where the same object has multiple possible references. For example, when there are multiple red cups in the scene, the system selects the most matching instance based on the object's position in the robot's current field of view, the user's gesture, or historical interaction records. Through the verification and correction by the rule engine, uncertainties and errors in the initial semantic frame are eliminated, generating a set of reliable and consistent candidate semantic frames, providing high-quality input for subsequent standardization processing.

[0024] In the above process, the rule engine can be combined with a knowledge graph to utilize the attributes (e.g., color, size, weight) and spatial relationships (e.g., located on a table, on the left side of a box) of objects in the scene to help resolve ambiguities. Furthermore, the rule engine can record verification logs, including the source of the completion and the basis for resolution, to enhance traceability. Through the cooperation of steps S2a and S2b, the system can robustly extract structured semantic information from natural language instructions. Even if the instructions are ambiguous, incomplete, or contradictory, it can generate reasonable candidate semantic frames, laying a solid foundation for subsequent standardization and verification.

[0025] Step S3: Standardize the candidate semantic frames to generate a standardized motion intent intermediate representation, and generate a unique hash value for the standardized motion intent intermediate representation.

[0026] In existing technologies, candidate semantic frames obtained from the initial parsing of upper-layer motion intentions often suffer from inconsistencies in their representation. For example, the same action may be described using multiple verbs such as "move," "translate," and "advance"; distance units may be mixed up using "centimeter," "inch," and "meter"; the coordinate system may be implicit in the context and not explicitly specified; object references may use aliases or abbreviations; control modes (e.g., position control, speed control, torque control) may not be explicitly declared, and so on. These inconsistencies can lead to subsequent processing modules failing to accurately understand the motion intentions and even generating incorrect execution instructions. Therefore, it is necessary to standardize the candidate semantic frames, transforming them into a unified and unambiguous Standardized Motion Intent Intermediate Representation (SMIIR), and generating a unique hash value for the Standardized Motion Intent Intermediate Representation. This scheme eliminates the diversity and ambiguity of natural language expressions by unifying verbs, units, coordinate systems, object references, and control modes. It ensures that the same motion intention can generate a consistent and standardized intermediate representation of motion intention regardless of the natural language input, thereby significantly improving the reproducibility and reliability of the system. At the same time, the generation of a unique hash value provides an immutable identification basis for subsequent verification, compilation, and auditing, enhancing the traceability of the system.

[0027] The standardization process adopted in this application includes the following five aspects: (1) Standardized Verbs: Verbs in candidate semantic frames are mapped to a predefined standardized verb library. The standardized verb library includes categories such as positional movement (e.g., move_to, move_along), posture adjustment (e.g., rotate_to, orient), grasping and releasing (e.g., grab, release), and force control operations (e.g., push_with_force, pull_with_force). The mapping process considers not only literal matching but also out-of-vocabulary words through a thesaurus and semantic similarity calculation. For example, “pick up” is mapped to “grasp”, “place” is mapped to “place”, and “move” is mapped to “move_along”. For verbs that cannot be directly mapped, the system records the mapping confidence and triggers manual confirmation or downgrade processing.

[0028] (2) Unify units: Unify physical quantities such as length, angle, force, and time in candidate semantic frames to the standard International System of Units (SI). For example, convert "10 inches" to "0.254 meters", "90 degrees" to "1.5708 radians", and keep "5 Newtons" unchanged. The unit conversion process needs to be combined with the scene context (e.g., the measurement habits of the robot's workspace) to ensure that the accuracy of the converted values ​​meets the control requirements.

[0029] (3) Unified Coordinate System: Based on the robot's current base frame, tool frame, and work object frame, the coordinate references in the candidate semantic frames are uniformly transformed. For example, if the instruction describes "move 10 cm to the left", the system needs to determine the "left" direction vector according to the current orientation of the robot's end effector and convert it into a displacement vector in the base frame. If no coordinate system is specified in the candidate semantic frame, the robot's base frame is used by default, and the coordinate system used is explicitly marked in the standardized intermediate representation so that the subsequent verification and execution modules can understand it accurately.

[0030] (4) Unified Object Reference: Map the object names in the candidate semantic frames to unique object identifiers (Object IDs) in the scene database. For example, "red cup" may correspond to "cup_red_001" in the scene. The mapping process needs to combine visual perception results, semantic segmentation labels, or a pre-registered list of objects. If multiple objects with the same name exist, disambiguation is performed based on the context (e.g., user gestures, object positions), or multiple candidates are generated and processed by subsequent verification steps.

[0031] (5) Unified Control Mode: Maps the implicit or explicit control modes (e.g., position control, speed control, torque control, impedance control, hybrid control) in candidate semantic frames to standardized control mode enumeration values. For example, "gently pick up" implies the use of impedance control mode and a lower stiffness setting; "quickly move to point A" implies the use of speed control mode. If no control mode is specified in the candidate semantic frame, it is automatically inferred based on the task type (e.g., grasp, place, push) and the default configuration.

[0032] After the above standardization process, the system generates a structured, standardized motion intent intermediate representation object. This object contains the following core fields: target pose, control mode, constraint set, temporal information, actuator reference, etc. This standardized motion intent intermediate representation has defined semantics and format, and can be consistently understood and processed by downstream verification and compilation modules.

[0033] Simultaneously, the system generates a unique hash value for this standardized campaign intent intermediate representation. The hash value is generated using a cryptographic hash function, taking the complete content of the standardized campaign intent intermediate representation (including all fields and metadata) as input and outputting a fixed-length digest string. This hash value has the following characteristics: ① Uniqueness: Different intermediate representations generate different hash values; ② Tamper-proof: Any modification to the intermediate representation will result in a change in the hash value; ③ Verifiability: The intermediate representation can be verified as unaltered by recalculating the hash value. The hash value will be used throughout the subsequent verification, compilation, and execution processes as a unique identifier for the campaign intent, for audit trails and anti-counterfeiting verification.

[0034] Furthermore, the standardization process for candidate semantic terms in step S3 may also include the following steps S3a and S3b: Step S3a: Map the verbs in the candidate semantic frames to a predefined standardized verb library.

[0035] As mentioned earlier, the standardized verb library contains classifications of common robot actions. In practical implementation, this standardized verb library can be stored in the form of configuration files or database tables, supporting dynamic expansion. The mapping algorithm can employ edit distance, WordNet semantic similarity, or cosine similarity based on word vectors. When a verb in a candidate semantic frame is not in the standardized verb library, the system calculates the closest standardized verb using semantic similarity and records the mapping confidence. For example, if the input verb is "move", the system calculates its similarity to "move_to" to be 0.92 and its similarity to "shift" to be 0.85, then selects "move_to" as the mapping result and records a confidence score of 0.92. If the confidence score is lower than a preset threshold, manual confirmation or a clarification request is triggered.

[0036] Step S3b: Based on the robot's current base coordinate system, tool coordinate system, and workpiece coordinate system, perform a unified transformation on the coordinate references in the candidate semantic frames to generate a globally consistent coordinate description.

[0037] Coordinate transformation involves calculating rigid body transformation matrices, including rotation and translation. The system obtains the current poses in each coordinate system from the robot controller or state server, and then transforms the target pose to a unified base coordinate system based on the reference coordinate system specified in the candidate semantic frame. If no coordinate system is specified in the candidate semantic frame, the robot base coordinate system is used by default, and the coordinate system used is explicitly labeled in the standardized motion intent intermediate representation so that the subsequent verification and execution modules can accurately understand it.

[0038] Step S4: Perform multi-dimensional validation on the standardized intermediate representation of motion intent.

[0039] In existing technologies, robot control instructions are typically written directly by programmers or generated via teach pendants and undergo rigorous testing before being put into production. However, when control instructions originate from natural language or AI models, their correctness and safety cannot be guaranteed. Directly executing unverified instructions may lead to serious consequences such as robot collisions, exceeding joint limits, and violating safety constraints. Therefore, it is crucial to conduct comprehensive, multi-dimensional verification of the standardized motion intention intermediate representation before execution. These multi-dimensional verifications can include pattern verification, kinematic feasibility verification, control pattern compatibility verification, constraint conflict verification, and temporal feasibility verification, among others.

[0040] This application's multi-dimensional verification examines the intermediate representation of standardized motion intent from five dimensions to ensure its feasibility, safety, and compliance in all aspects. The verification for each dimension is described in detail below.

[0041] 1) Pattern Validation: Check whether the structure of the intermediate representation of the standardized motion intent conforms to the predefined schema. For example, verify whether the required fields (e.g., target pose, control mode) are complete, whether the field types are correct (e.g., pose should be a 6-DOF floating-point array), and whether the value range is reasonable (e.g., joint angles should be within [...]). -π , π (Within the specified range). Pattern validation is equivalent to syntax checking, ensuring that the intermediate representation itself is valid and complete.

[0042] 2) Kinematic Feasibility Verification: Based on the robot's forward and inverse kinematics models, the system calculates whether the target pose is within joint angle limits and the workspace, and assesses singularity risk. Specifically, the system first converts the Cartesian-space-based target pose into angle values ​​in joint space using an inverse kinematics solver, and then checks whether each joint angle is within its physical limits (e.g., joint 1: -170°~170°). Simultaneously, it calculates whether the target pose is within the robot's reachable workspace (i.e., whether an inverse solution exists). Furthermore, singularity risk is assessed, specifically whether the condition number of the Jacobian matrix is ​​too large. If it approaches a singular configuration, a warning or suggested alternative pose is generated. If the kinematics are infeasible, a verification failure report is generated, including the reasons for infeasibility and recommended alternative poses.

[0043] 3) Control Mode Compatibility Verification: This involves checking whether the control mode specified in the standardized motion intent intermediate representation matches the capabilities of the current actuator and the task phase. For example, if the actuator is a common servo motor without torque control capabilities, torque control mode cannot be used; if the task phase involves high-speed movement, position control or speed control mode should be used, rather than impedance control. This application defines the compatibility relationships between various modes by constructing a control mode compatibility matrix and makes judgments accordingly.

[0044] 4) Constraint Conflict Verification: Checks whether the intended movement violates predefined constraints. Constraints are categorized by priority into safety constraints (e.g., maximum force not exceeding 100N), hard constraints (e.g., no entry into safety restricted areas), soft constraints (e.g., try to get as close to the target point as possible), and preference constraints (e.g., preferential use of green fixtures). The system checks each level of constraints sequentially to ensure they are met and takes appropriate measures based on any violations.

[0045] 5) Timing Feasibility Verification: This checks whether the intended motion can be completed within the specified time window. The system estimates the execution time based on parameters such as motion distance, speed limits, and acceleration limits, and compares this time with the deadline in the instruction. If the motion cannot be completed on time, it is marked as timing infeasible, and a delay estimation report is generated.

[0046] The five dimensions of verification mentioned above are independent yet complementary, collectively constituting a comprehensive security check of the motion intent. Only when all verifications pass is the subsequent compilation and execution phase permitted. If any verification fails, the system will execute a rejection, downgrade, or clarification process based on the failure type.

[0047] The aforementioned multi-dimensional verification further includes verification of actuator capabilities, namely, checking whether the motion parameters (e.g., velocity, torque, acceleration) required by the intermediate representation of the standardized motion intention are within the current physical capabilities of the actuator. It also includes runtime capability space projection verification, which projects the motion intention onto the robot's real-time dynamics and kinematic capability space to determine whether it can be executed under the current system load, temperature, and energy consumption conditions.

[0048] In the multi-dimensional verification of the aforementioned examples, constraint conflict verification is a particularly crucial step. In existing technologies, constraints in robotic systems are typically treated as a whole; if any constraint is violated, the system directly refuses to execute. While this ensures safety, it leads to a low task success rate, especially in flexible manufacturing and human-robot collaboration scenarios, where minor violations of certain non-critical constraints could be tolerated or downgraded. For example, in the task of "gently picking up a red cup," if safety constraints (e.g., maximum force not exceeding 50N) are strictly adhered to, but soft constraints (e.g., expected contact point position deviation less than 5mm) are not met, a direct refusal by the system would cause task interruption, even though this deviation would not actually pose a safety issue. Therefore, it is necessary to classify constraints to achieve a balance between safety and task completion rate.

[0049] The constraint conflict verification in this embodiment adopts a hierarchical constraint processing mechanism. The constraint conflict verification can be implemented through steps S4a to S4d, as detailed below: Step S4a: Divide the constraints into four levels according to priority: safety constraints, hard constraints, soft constraints, and preference constraints.

[0050] Safety constraints are inviolable limits, such as maximum force limits, maximum speed limits, and safety restriction zone boundaries. Violating safety constraints will directly lead to personal injury or equipment damage. Hard constraints are necessary conditions for task execution, such as the target pose must be within the workspace or the gripper opening degree must match the object size. Violating hard constraints will prevent the task from being completed. Soft constraints are desired but not necessary conditions, such as getting as close to the target center as possible or minimizing movement time. Violating soft constraints will affect task quality but will not cause danger. Preference constraints are the user's subjective preferences, such as prioritizing the use of a specific gripper or choosing a particular path. Violating preference constraints only affects the user experience.

[0051] Step S4b: Check in turn whether the constraints at each level are satisfied.

[0052] The system checks constraints in the following order: safety constraints, hard constraints, soft constraints, and preference constraints. This hierarchical design ensures that the highest priority safety constraints are checked first, and any violation immediately terminates subsequent checks, saving computational resources.

[0053] Step S4c: If the security constraint or hard constraint is violated, the verification is directly determined to fail and the corresponding exception code is generated.

[0054] If the safety constraint "maximum force not exceeding 50N" is violated, for example, if the motion intent requires an 80N force, the system immediately generates the exception code "SAFETY_FORCE_EXCEEDED" and terminates the verification process. If the hard constraint "target pose is within the workspace" is violated, for example, if the target point is outside the reachable range of the robot arm, the system generates the exception code "HARD_OUT_OF_WORKSPACE". These exception codes will be logged and passed to subsequent rejection or clarification modules.

[0055] Step S4c: If a soft constraint or preference constraint is violated, record the violation information and allow degraded execution.

[0056] For example, if the soft constraint "target point deviation less than 5mm" is violated (the actual deviation is 8mm), the system records the violation details (deviation value 8mm, allowable threshold 5mm), but still allows the motion intention to proceed to the subsequent compilation and execution stages. For violations of preference constraints, the system only generates an alarm log, without affecting execution. This hierarchical handling approach allows the system to maximize the flexibility and success rate of task completion while ensuring absolute safety.

[0057] As can be seen from the constraint conflict verification examples in steps S4a to S4d above, by dividing constraints into four priority levels—safe, hard, soft, and preference—and taking differentiated treatment for different levels (i.e., direct failure for violations of safe constraints and / or hard constraints, and allowing degraded execution for violations of soft constraints and / or preference constraints), a balance between safety and task completion rate is achieved, improving the system's practicality and robustness.

[0058] The degraded execution described in the above embodiments can be as follows: when a soft constraint is violated, the details of the soft constraint violation are marked in the compilation certificate, and low-level motion commands are allowed to be issued normally; when a preference constraint is violated, an alarm log is generated, but execution is still allowed. The soft constraint violation marker in the compilation certificate can be used for subsequent auditing and task review, allowing operators to understand the quality loss during task execution. The alarm log records the violation information of preference constraints for system optimization and user preference learning.

[0059] In multi-dimensional verification, control mode compatibility verification is another crucial step in ensuring the safe operation of robots. In existing technologies, control mode selection is typically statically specified by programmers during code writing or manually set by operators in the teach pendant. However, when control instructions originate from natural language or AI models, the specified control mode may not match the actual capabilities of the actuator or the current task stage. For example, requiring a standard servo motor without torque control capabilities to execute torque control mode, or requiring impedance control mode to be used during high-speed movement. Directly executing incompatible control modes may lead to actuator malfunction, damage, or even injury. Therefore, it is necessary to perform control mode compatibility verification during the instruction compilation stage.

[0060] The control mode compatibility verification performed in step S4 of this application adopts an intelligent verification and automatic adaptation mechanism based on a compatibility matrix, which is mainly achieved through the following steps S4e to S4g: Step S4e: Construct a control mode compatibility matrix, which defines the compatibility relationships between position control, speed control, torque control, impedance control, and hybrid control modes.

[0061] The control mode compatibility matrix is ​​a two-dimensional table where rows and columns represent the source and target control modes, respectively. Matrix elements indicate the compatibility level between the two modes (e.g., "fully compatible," "conditionally compatible," "incompatible"). For example, position control and speed control are usually compatible because many robot controllers support setting speed limits in position control mode; however, torque control and position control may be incompatible in some systems because torque control requires a dedicated current loop configuration. The compatibility matrix is ​​constructed based on the robot manufacturer's technical specifications, controller firmware version, and historical operating data, and can be dynamically updated with system upgrades.

[0062] Step S4f: Based on the control mode field in the intermediate representation of standardized motion intent, determine whether the selected mode matches the current actuator capability and task stage by querying the control mode compatibility matrix.

[0063] The system first reads the control mode field (e.g., "IMPEDANCE") from the standardized motion intention intermediate representation, then obtains the current actuator capability description (e.g., "Supports position control, speed control, impedance control, but does not support torque control") from the robot state server, and finally obtains the current task stage (e.g., "High-speed movement stage") from the task scheduler. Combining this information, the system queries the control mode compatibility matrix to determine if the specified control mode is feasible. For example, if "IMPEDANCE" is specified, and the actuator supports impedance control, and the current task stage is "Fine Operation Stage" (suitable for impedance control), then the verification passes; if the current task stage is "High-speed movement Stage" (generally unsuitable for impedance control), then the verification fails.

[0064] Step S4g: If it is determined that the selected mode does not match the current executor capability and task stage, a compatible alternative mode is automatically selected and the standardized motion intention intermediate representation is updated.

[0065] If the user command specifies the "torque control" mode, but the actuator does not support it, the system automatically queries the control mode compatibility matrix to find the closest alternative mode that the actuator supports, such as "impedance control." Then, it updates the control mode field in the standardized motion intention intermediate representation to "IMPEDANCE" and records the mode switch. This automatic adaptation mechanism avoids execution failures due to mode mismatch, improving the system's robustness and ease of use.

[0066] The control mode compatibility verification in the examples of steps S4e to S4g above achieves intelligent adaptation of control modes by constructing a control mode compatibility matrix, clearly defining the compatibility relationships between various modes, and automatically selecting a compatible alternative mode and updating the intermediate representation when a mismatch is detected. This avoids human error and improves the system's adaptive capability.

[0067] Furthermore, the control mode compatibility matrix can also include safety parameter thresholds for each control mode, including maximum speed, maximum torque, and maximum impedance stiffness. Different control modes have different safe operating parameter ranges. For example, in position control mode, the maximum speed may be limited to 1 m / s; while in torque control mode, the maximum torque may be limited to 20 Nm. When the system automatically selects an alternative mode, it needs to simultaneously adjust the safety parameters in the standardized motion intention intermediate representation to within the safety parameter threshold range to ensure safe operation after mode switching. Simultaneously with automatic selection of the alternative mode, the safety parameters in the standardized motion intention intermediate representation are adjusted to within the safety parameter threshold range, and a mode switching log is recorded. For example, when switching from torque control to impedance control, the system adjusts the maximum torque threshold from 30 Nm to 20 Nm and sets the impedance stiffness to the default value of 500 N / m, while recording a log entry: "Mode Switch: TORQUE → IMPEDANCE, Safety Parameters Adjusted, Switching Time: 2026-06-24 10:30:00.123". The linkage adjustment mechanism in the above embodiment embeds the safety parameter threshold corresponding to each mode in the control mode compatibility matrix and synchronously adjusts the safety parameters in the intermediate representation to the threshold range when automatically switching modes. This ensures that the mode remains in a safe operating range after switching, and records the switching log for easy traceability, thus realizing adaptive linkage between mode switching and safety parameters.

[0068] Step S5: After successful verification, construct a motion graph based on the standardized intermediate representation of motion intent, compile the motion graph into low-level motion instructions, and generate a compilation certificate containing hash values, compilation results, and model version.

[0069] In existing technologies, the conversion from high-level task descriptions to low-level execution instructions typically relies on programmers manually writing code or using teach pendants for point-by-point programming. This approach is inefficient and prone to errors. Another common practice is to directly call API functions provided by robot manufacturers, passing in task parameters directly. However, this approach lacks systematic management of temporal dependencies and spatial interferences between multi-step tasks, easily leading to execution conflicts. Furthermore, the execution process lacks traceable records, making it difficult to pinpoint the cause of problems. This application addresses these issues by constructing a motion graph. Specifically, after successful verification, a motion graph is constructed based on a standardized intermediate representation of motion intentions, and the motion graph is compiled into low-level motion instructions, generating a compilation certificate containing hash values, compilation results, and model versions. A motion graph is a directed acyclic graph used to organize and coordinate the execution order and dependencies between multiple motion intentions. The above solution, by constructing a motion graph, organizes multiple motion intentions into a structured execution plan with temporal dependencies and spatial constraints, effectively coordinating the collaborative actions of multiple steps or multiple actuators and avoiding execution conflicts. The motion graph is compiled into low-level motion instructions, achieving a seamless conversion from high-level semantics to executable instructions. Simultaneously, a compilation certificate containing hash values, compilation results, and model versions is generated, providing a complete and auditable chain of evidence for each execution, facilitating subsequent traceability, analysis, and system iteration.

[0070] In existing technologies, multi-step robotic tasks typically employ sequential programming, where programmers predefine the execution order and parameters of each step. This approach lacks flexibility in the face of dynamic environments or task changes and struggles to handle parallel relationships and resource contention between steps. For example, in a dual-arm collaborative operation scenario, both arms need to grasp different objects simultaneously. If sequential programming is used, one arm must wait for the other to complete before it can begin, leading to inefficiency. Furthermore, when spatial interference exists between task steps (e.g., overlapping workspaces of the two arms), sequential programming cannot automatically detect and avoid conflicts. Therefore, a graph structure method is needed that can automatically organize the execution order of multiple steps and coordinate resource conflicts.

[0071] The motion graph constructed in step S5 of this application is precisely to solve the above-mentioned problems. As an embodiment of this application, the construction of the motion graph based on the standardized intermediate representation of motion intent is further refined into the following steps S5a to S5c: Step S5a: Map each standardized motion intention intermediate representation to a node in the motion graph. The node attributes include target pose, control mode, desired force, and time window.

[0072] Each node represents an atomic motion intention, such as "move to point A," "grab an object," "rotate your wrist," and so on. The node's attributes are directly extracted from the standardized intermediate representation of the motion intention: target pose (target_pose) is a 6-DOF Cartesian coordinate system or an array of joint angles; control mode (control_mode) is an enumerated value (POSITION / VELOCITY / TORQUE / IMPEDANCE / HYBRID); desired force (desired_force) is a 3D force vector or scalar; and the time window (time_window) includes the earliest start time (earliest_start), latest end time (latest_end), and estimated duration (estimated_duration). These attributes provide the foundational data for subsequent dependency analysis and scheduling.

[0073] Step S5b: Based on the temporal dependency and spatial interference conditions, add directed edges between nodes, with the directed edges carrying transition conditions or synchronization signals.

[0074] In this embodiment, temporal dependency refers to the requirement that certain nodes must complete before they can begin; for example, "grabbing" must occur after "moving to the grabbing position". Spatial interference refers to the possibility of collisions or resource conflicts between two nodes during execution; for example, two robotic arms accessing the same area simultaneously. For temporal dependencies, directed edges are added and transition conditions are set, such as "the previous node has completed and its state is successful". For spatial interference conditions, synchronization signal edges are added, requiring the relevant nodes to stagger in time or coordinate through mutex locks. Transition conditions and synchronization signals can be represented by logical expressions, such as "gripper_state == closed && force_sensor". <threshold”。

[0075] Step S5c: Perform topological sorting on the motion graph to generate a sequence of instructions that can be executed sequentially or in parallel.

[0076] Topological sorting algorithms, such as the Kahn algorithm or DFS-based algorithms, can be used to arrange the nodes in a DAG into a linear sequence, such that for every directed edge (u, v), node u always appears before node v in the sequence. Nodes without dependencies can be adjacent in the sequence and marked as executable in parallel. For example, in two independent tasks, "picking up a cup" and "opening a drawer," if they do not share any actuators and their workspaces do not overlap, they can be scheduled to execute in parallel within the same time slice. The generated instruction sequence contains the execution parameters, trigger conditions, and synchronization signals for each instruction and can be directly sent to the underlying motion controller.

[0077] Furthermore, generating a sequence of instructions that can be executed sequentially or in parallel may include: assigning a deadline timestamp to each instruction and checking for timing conflicts; if a timing conflict exists, triggering rescheduling or marking a deadline violation. The assignment of deadline timestamps is based on the node's time window attribute: the latest end time (latest_end) of each instruction is used as its deadline timestamp. Timing conflict checks are performed through simulated execution: the system simulates the execution of each instruction sequentially according to the topology sequence and accumulates the time. If the simulated end time of an instruction exceeds its deadline timestamp, a timing conflict is determined. In this case, the system has two options: one is to trigger rescheduling, i.e., attempting to adjust motion parameters (e.g., increasing speed, shortening the path) to meet the deadline; the other is to mark a deadline violation, i.e., recording the violation information in the compilation certificate, but still allowing execution, with the real-time control system deciding at runtime whether to force execution or abort. This mechanism ensures that even under time-pressured conditions, the system can make reasonable decisions rather than simply refusing execution.

[0078] During the compilation process, the system simultaneously generates a compilation certificate containing a hash value, compilation results, and model version. The compilation certificate is a structured digital document that records complete information about this compilation. The hash value, derived from the hash value of the standardized intermediate representation of the motion intent generated in step S3, is used to uniquely associate the compilation result with the original motion intent. The compilation result includes whether the verification passed, the number of motion graph nodes, and the length of the instruction sequence. The model version records the version number of the AI ​​model or rule engine that generated the motion intent, such as "LLM-v2.1" or "RuleEngine-20260601". The existence of the compilation certificate ensures that each execution has a complete and auditable chain of evidence, facilitating post-execution traceability, troubleshooting, and system optimization.

[0079] In multi-dimensional verification, kinematic feasibility verification is crucial to ensuring that the robot can physically reach the target pose. In existing technologies, kinematic verification is typically performed manually by engineers during offline programming or confirmed through trial and error during teaching. This approach is not only time-consuming but also prone to overlooking certain extreme positions or singularities, leading to problems such as joint over-limits and singular vibrations during actual operation. For example, when a natural language instruction tells the robot to "reach its hand to the corner of the table," this position might be located at the edge of the robot's workspace or even beyond its reach. Executing this instruction directly without verification could result in joint collisions or motor overloads. Therefore, it is necessary to automatically perform kinematic feasibility verification during the instruction compilation phase.

[0080] The kinematic feasibility verification in the above embodiments employs a systematic verification method based on forward and inverse kinematic models. Specifically, the kinematic feasibility verification can be achieved through the following steps S5d and S5e: Step S5d: Based on the robot's forward and inverse kinematics models, calculate whether the target pose is within the joint angle limits and workspace, and assess the singularity risk.

[0081] Forward kinematics (FK) is used to calculate the pose of the robot's end effector given joint angles; inverse kinematics (IK) is used to solve for the corresponding joint angles given the end effector pose. The system first reads the target pose (usually position and orientation in Cartesian coordinates) from a standardized intermediate representation of motion intent, and then calls the IK solver to calculate whether a set of joint angles exists that allows the end effector to reach that pose. The IK solver can use analytical methods (suitable for specific robot configurations, such as 6-axis serial robots) or numerical methods (e.g., Jacobian pseudo-inverse method, damped least squares method). If the IK solution is successful, it further checks whether each joint angle is within its physical limits (e.g., joint 1: -170°~170°, joint 2: -90°~90°, etc.). If all joint angles are within their limits, the target pose is reachable within the workspace; otherwise, it is determined to be kinematically infeasible.

[0082] In addition to reachability checks, singularity risk also needs to be assessed. Singularity refers to the decrease in the rank of the Jacobian matrix when the robot is in certain special configurations, causing the end effector to lose its ability to move in certain directions or require extremely high joint velocities. The system determines whether it is close to a singular configuration by calculating the condition number or the minimum singular value of the Jacobian matrix. If the condition number is greater than a preset threshold, a high singularity risk is considered to exist, and a warning message is generated.

[0083] Step S5e: If the singularity risk assessment determines that the kinematics is not feasible, then generate a verification failure report containing the reasons for the infeasibility and recommended alternative poses.

[0084] For example, if the target pose is outside the workspace, the system reports "Target pose (0.8, 0.2, 0.5) is outside the workspace, the nearest reachable pose is (0.75, 0.2, 0.5)", and provides a recommended alternative pose. If there is a singularity risk, the system reports "A singular configuration exists near the target pose; it is recommended to fine-tune the attitude angle to (0.5°, 0.2°, 0.1°) to avoid singularity." These recommended alternative poses can be obtained by sampling a small area near the target pose and re-solving the IK algorithm. Verification failure reports will be passed to the rejection or clarification module for reference by the user or upstream system.

[0085] The compilation certificate generated in step S5 is a crucial document for ensuring the traceability and auditability of instructions. In existing technologies, the execution records of control instructions are typically stored in log files, which are easily tampered with or forged, lack reliable timestamps and anti-tampering mechanisms, and cannot meet the stringent post-audit and legal compliance requirements of industrial scenarios. For example, in fields such as medical robotics, precision surgery, or high-end manufacturing, every operation requires irrefutable evidence to prevent unclear liability determination or malicious tampering.

[0086] To address the aforementioned issues, this application introduces digital signatures and blockchain-based evidence storage mechanisms into the compiler certificate. As one embodiment of this application, the compiler certificate further includes a digital signature and a trusted timestamp. The digital signature is based on a private key to sign the hash value and the compilation result. The trusted timestamp is issued by a trusted timestamp service provider. The compiler certificate is written to at least one of a local immutable log, a trusted execution environment, a remote audit server, or a distributed ledger for subsequent auditing and tamper-proof verification.

[0087] The digital signature generation process is as follows: The system uses its private key to sign key fields in the compiled certificate (including hash value, compilation result, model version, timestamp, etc.), generating a digital signature value. The public key can be publicly released, and anyone can use it to verify the authenticity of the signature. Thus, once the compiled certificate is issued, any modification to the certificate content will cause signature verification to fail, thereby ensuring the integrity of the certificate.

[0088] Trusted timestamps are issued by trusted timestamp service providers (such as legal timestamp service providers or third-party TSAs) to ensure the legal validity of the compilation certificate's generation time. When generating a compilation certificate, the system sends the certificate's hash value to the TSA, which returns a signed response with a timestamp, which is then appended to the compilation certificate.

[0089] Finally, the compiler certificate is written to a blockchain or trusted execution environment. A blockchain provides a decentralized, immutable distributed ledger, where each compiler certificate is recorded as a transaction, and anyone can query and verify it. A trusted execution environment (e.g., Intel SGX, ARM TrustZone) provides an isolated execution environment, ensuring that the storage and access of certificates are not interfered with by malware. Through this dual protection, compiler certificates possess extremely high tamper resistance and trustworthiness, providing strong technical support for auditing, liability determination, and compliance in industrial scenarios.

[0090] Although multi-dimensional verification filters out unsafe or unreasonable motion intentions as much as possible, verification failures may still occur in actual operation due to safety constraints or insufficient hardware capabilities. For example, when the robot detects excessively high joint temperatures or unstable power supply voltage, the system may refuse to execute the motion intention, even if it is kinematically feasible, for safety reasons. In existing technologies, simply refusing execution leads to task interruption, while blind execution carries risks; there is a lack of an intelligent degradation scheme that can both ensure safety and maintain system operation.

[0091] To address the aforementioned issues, this application provides a security degradation path mechanism, as one embodiment of this application. Figure 1 The example method may also include: when multi-dimensional verification fails due to security constraints or insufficient hardware capabilities, triggering a security degradation path, wherein the security degradation path may be to replace the target pose in the intermediate representation of motion intent with a preset safe pose, switch the control mode to a low-power mode, generate a degradation execution certificate, and the degradation execution certificate contains the original failure reason, degradation parameters and recovery conditions for subsequent manual confirmation or automatic recovery.

[0092] Specifically, when verification fails due to a violation of safety constraints (e.g., exceeding the maximum force limit) or insufficient hardware capabilities (e.g., joint actuator overheating), the system does not directly terminate the task but instead enters a safety degradation mode. First, the system selects a suitable pose from a pre-configured safe pose library (e.g., the robot returning to its home pose or moving to a standby position away from the operator) and replaces the target pose in the intermediate representation of motion intent with this safe pose. Second, the control mode is switched to a low-power mode (e.g., limiting speed to 20% of normal and torque to 50%) to reduce energy consumption and heat generation. Then, a degradation execution certificate is generated. This certificate, in addition to containing information from the original compilation certificate, also records the original failure reason (e.g., "Safety constraint: maximum force 50N, actual 60N"), degradation parameters (e.g., "Target pose has been replaced with home_pose, speed limit 0.2m / s"), and recovery conditions (e.g., "Normal mode can be restored after joint temperature drops below 45°C"). The degradation execution certificate can also include a digital signature and timestamp to ensure its credibility. In this way, the system can continue to operate safely and controllably when faced with abnormal situations, avoiding complete task failure, while retaining the means for subsequent manual confirmation or automatic recovery.

[0093] From the above Figure 1The robot control instruction compilation method in the example demonstrates several advantages. First, by acquiring the upper-level motion intent and generating candidate semantic frames containing semantic information such as verbs, objects, modifiers, time, and force parameters, structured parsing of ambiguous or incomplete natural language instructions is achieved, providing an accurate semantic foundation for subsequent processing. Furthermore, by standardizing the candidate semantic frames, verbs, units, coordinate systems, object references, and control modes are unified, and a unique hash value is generated, eliminating ambiguity caused by different representations and ensuring the consistency and traceability of the intermediate representation of motion intent. Second, by performing multi-dimensional verification including pattern verification, kinematic feasibility verification, control mode compatibility verification, constraint conflict verification, and temporal feasibility verification, the safety, feasibility, and compliance of motion intent can be comprehensively evaluated during the instruction compilation stage, effectively preventing the robot from executing dangerous or infeasible actions and significantly improving the system's robustness and safety. Third, after successful verification, a motion graph is constructed based on the standardized intermediate representation of motion intent and compiled into low-level motion instructions. Simultaneously, a compilation certificate containing hash values, compilation results, and model versions is generated, achieving end-to-end automated conversion from natural language to executable instructions and providing complete execution credentials for subsequent auditing and system optimization. In summary, the technical solution of this application overcomes the ambiguity, security risks and lack of traceability of the direct parsing of natural language instructions in the prior art, and realizes efficient, safe and verifiable robot control instruction compilation.

[0094] Please see the appendix Figure 2 This application provides a robot control instruction compilation device, which includes an acquisition module 201, a first generation module 202, a second generation module 203, a verification module 204, and a compilation module 205, as detailed below: The acquisition module 201 is used to acquire the upper-level motion intention, wherein the upper-level motion intention originates from at least one of natural language instructions, human-computer interaction interface, AI strategy model, visual language model, task planner or script interface; The first generation module 202 is used to generate one or more candidate semantic frames based on the upper-layer motion intention, wherein the candidate semantic frames contain semantic information of verbs, objects, modifiers, time and force parameters; The second generation module 203 is used to standardize the candidate semantic frames, generate a standardized motion intention intermediate representation, and generate a unique hash value for the standardized motion intention intermediate representation. The standardization process includes unifying verbs, units, coordinate systems, object references, and control modes. The verification module 204 is used to perform multi-dimensional verification on the intermediate representation of standardized motion intentions. The multi-dimensional verification includes pattern verification, kinematic feasibility verification, control pattern compatibility verification, constraint conflict verification, and temporal feasibility verification. The compilation module 205 is used to construct a motion graph based on the standardized intermediate representation of motion intent after verification, compile the motion graph into low-level motion instructions, and generate a compilation certificate containing hash values, compilation results and model version.

[0095] From the above Figure 2 As can be seen from the example robot control instruction compilation device, on the one hand, by acquiring the upper-level motion intention and generating candidate semantic frames containing semantic information such as verbs, objects, modifiers, time and force parameters, the structured parsing of ambiguous or incomplete natural language instructions is realized, providing an accurate semantic foundation for subsequent processing. By standardizing the candidate semantic frames, verbs, units, coordinate systems, object references and control modes are unified, and a unique hash value is generated, eliminating ambiguity caused by different expression methods and ensuring the consistency and traceability of the intermediate representation of motion intention. On the other hand, by performing multi-dimensional verification including pattern verification, kinematic feasibility verification, control mode compatibility verification, constraint conflict verification and temporal feasibility verification, the safety, feasibility and compliance of motion intention can be comprehensively evaluated in the instruction compilation stage, effectively avoiding the robot from executing dangerous or infeasible actions, and significantly improving the robustness and safety of the system. On the third hand, after the verification is passed, a motion graph is constructed based on the standardized intermediate representation of motion intention and compiled into low-level motion instructions. At the same time, a compilation certificate containing hash value, compilation result and model version is generated, realizing end-to-end automated conversion from natural language to executable instructions, and providing complete execution credentials, which facilitates subsequent auditing and system optimization. In summary, the technical solution of this application overcomes the ambiguity, security risks and lack of traceability of the direct parsing of natural language instructions in the prior art, and realizes efficient, safe and verifiable robot control instruction compilation.

[0096] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example... Figure 3 As shown, the electronic device 3 in this embodiment mainly includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a program for compiling robot control instructions. When the processor 30 executes the computer program 32, it implements the steps described in the robot control instruction compilation method embodiment, for example... Figure 1 Steps S1 to S5 are shown. Alternatively, when processor 30 executes computer program 32, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 2 The functions of the acquisition module 201, the first generation module 202, the second generation module 203, the verification module 204, and the compilation module 205 are shown.

[0097] For example, the computer program 32 of the robot control instruction compilation method mainly includes: obtaining upper-level motion intentions, wherein the upper-level motion intentions originate from at least one of natural language instructions, human-computer interaction interfaces, AI strategy models, visual language models, task planners, or script interfaces; generating one or more candidate semantic frames based on the upper-level motion intentions, wherein the candidate semantic frames contain semantic information of verbs, objects, modifiers, time, and force parameters; performing standardization processing on the candidate semantic frames to generate a standardized intermediate representation of motion intentions, and generating a unique hash value for the standardized intermediate representation of motion intentions, wherein the standardization processing includes unifying verbs, units, coordinate systems, object references, and control modes; performing multi-dimensional verification on the standardized intermediate representation of motion intentions, wherein the multi-dimensional verification includes mode verification, kinematic feasibility verification, control mode compatibility verification, constraint conflict verification, and temporal feasibility verification; after the verification is passed, constructing a motion graph based on the standardized intermediate representation of motion intentions, compiling the motion graph into low-level motion instructions, and generating a compilation certificate containing a hash value, compilation results, and model version. Computer program 32 can be divided into one or more modules / units, which are stored in memory 31 and executed by processor 30 to complete this application. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, describing the execution process of computer program 32 in electronic device 3. For example, computer program 32 can be divided into the functions of an acquisition module 201, a first generation module 202, a second generation module 203, a verification module 204, and a compilation module 205 (a module in a virtual device). The specific functions of each module are as follows: Acquisition module 201 is used to acquire upper-level motion intentions, wherein the upper-level motion intentions originate from at least one of natural language instructions, human-computer interaction interfaces, AI strategy models, visual language models, task planners, or script interfaces; the first generation module 202 is used to generate one or more candidate semantic frames based on the upper-level motion intentions, wherein the candidate semantic frames contain semantic information of verbs, objects, modifiers, time, and force parameters; the second generation module 203 is used to... Candidate semantic frames are standardized to generate a standardized intermediate representation of motion intent, and a unique hash value is generated for the standardized intermediate representation of motion intent. The standardization process includes unifying verbs, units, coordinate systems, object references, and control modes. The verification module 204 is used to perform multi-dimensional verification on the standardized intermediate representation of motion intent. The multi-dimensional verification includes mode verification, kinematic feasibility verification, control mode compatibility verification, constraint conflict verification, and temporal feasibility verification. The compilation module 205 is used to construct a motion graph based on the standardized intermediate representation of motion intent after the verification is passed, and compile the motion graph into low-level motion instructions. At the same time, a compilation certificate containing hash value, compilation result, and model version is generated.

[0098] Electronic device 3 may include, but is not limited to, processor 30 and memory 31. Those skilled in the art will understand that... Figure 3 This is merely an example of electronic device 3 and does not constitute a limitation on electronic device 3. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0099] The processor 30 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0100] The memory 31 can be an internal storage unit of the electronic device 3, such as a hard disk or RAM. The memory 31 can also be an external storage device of the electronic device 3, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 31 can include both internal and external storage units of the electronic device 3. The memory 31 is used to store computer programs and other programs and data required by the electronic device. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0101] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed. That is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above-described device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0102] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0103] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0104] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0105] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0106] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0107] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a storage medium. Based on this understanding, all or part of the processes in the above-described embodiments can also be implemented by a computer program instructing related hardware. The computer program for the robot control instruction compilation method can be stored in a storage medium. When executed by a processor, this computer program can implement the steps of the various method embodiments described above, namely, obtaining the upper-level motion intention, wherein the upper-level motion intention originates from at least one of natural language instructions, human-computer interaction interfaces, AI strategy models, visual language models, task planners, or script interfaces; and generating one or more candidate semantic frames based on the upper-level motion intention, wherein the candidate semantic frames contain verbs, objects, modifiers, time, and force parameters. The process involves: 1) standardizing candidate semantic frames to generate a standardized intermediate representation of motion intent, and generating a unique hash value for this intermediate representation. Standardization includes unifying verbs, units, coordinate systems, object references, and control modes. 2) Performing multi-dimensional verification on the standardized intermediate representation of motion intent, including mode verification, kinematic feasibility verification, control mode compatibility verification, constraint conflict verification, and temporal feasibility verification. 3) After successful verification, constructing a motion graph based on the standardized intermediate representation of motion intent, compiling the motion graph into low-level motion instructions, and generating a compilation certificate containing the hash value, compilation result, and model version. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate form. The storage medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, external hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the contents of the storage medium may be appropriately added to or subtracted from the contents according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the storage medium may not include electrical carrier signals and telecommunication signals.

[0108] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application. The specific embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the protection scope of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for compiling robot control instructions, characterized in that, The method includes: Step S1: Obtain the upper-level motion intention, which originates from at least one of natural language commands, human-computer interaction interfaces, AI strategy models, visual language models, task planners, or script interfaces; Step S2: Based on the upper-layer motion intent, generate one or more candidate semantic frames, wherein the candidate semantic frames contain semantic information of verbs, objects, modifiers, time and force parameters; Step S3: Standardize the candidate semantic frames to generate a standardized motion intent intermediate representation and generate a unique hash value for the standardized motion intent intermediate representation. The standardization process includes unifying verbs, units, coordinate systems, object references and control modes. Step S4: Perform multi-dimensional verification on the standardized motion intention intermediate representation. The multi-dimensional verification includes mode verification, kinematic feasibility verification, control mode compatibility verification, constraint conflict verification, and temporal feasibility verification. The constraint conflict verification includes: dividing constraints into four levels according to priority: safety constraints, hard constraints, soft constraints, and preference constraints; checking whether each level of constraints is satisfied in turn; if the safety constraint or the hard constraint is violated, the verification is directly determined to be unsuccessful and a corresponding exception code is generated; if the soft constraint or the preference constraint is violated, the violation information is recorded and downgraded execution is allowed. The control mode compatibility verification includes: constructing a control mode compatibility matrix, which defines the compatibility relationship between position control, speed control, torque control, impedance control, and hybrid control modes; based on the control mode field in the standardized motion intention intermediate representation, judging whether the selected mode matches the current actuator capability and task stage by querying the control mode compatibility matrix; if it is determined that the selected mode does not match the current actuator capability and task stage, a compatible alternative mode is automatically selected and the standardized motion intention intermediate representation is updated. Step S5: After verification, construct a motion graph based on the standardized motion intention intermediate representation, and compile the motion graph into low-level motion instructions. At the same time, generate a compilation certificate containing the hash value, compilation result, and model version. The construction of the motion graph based on the standardized motion intention intermediate representation includes: mapping each standardized motion intention intermediate representation to a node in the motion graph. The node attributes include target pose, control mode, expected force, and time window. According to the temporal dependency and spatial interference conditions, add directed edges between nodes. The directed edges carry transition conditions or synchronization signals. Perform topological sorting on the motion graph to generate a sequence of instructions that can be executed sequentially or in parallel.

2. The robot control instruction compilation method according to claim 1, characterized in that, The control mode compatibility matrix also includes safety parameter thresholds for each control mode, including maximum speed, maximum torque, and maximum impedance stiffness. While automatically selecting the alternative mode, the safety parameters in the standardized motion intention intermediate representation are adjusted to the range of the safety parameter thresholds, and the mode switching log is recorded.

3. The robot control instruction compilation method according to claim 1, characterized in that, The compilation certificate also includes a digital signature and a trusted timestamp. The digital signature is based on a private key to sign the hash value and the compilation result. The trusted timestamp is issued by a trusted timestamp service provider. The compilation certificate is written into a blockchain or trusted execution environment for subsequent auditing and tamper-proof verification.

4. The robot control instruction compilation method according to claim 1, characterized in that, The method further includes: When the multi-dimensional verification fails due to security constraints or insufficient hardware capabilities, a security degradation path is triggered. The security degradation path includes: replacing the target pose in the intermediate representation of the motion intention with a preset safe pose, and switching the control mode to a low-power mode. A downgrade execution certificate is generated, which includes the original failure reason, downgrade parameters, and recovery conditions, for subsequent manual confirmation or automatic recovery.

5. A robot control instruction compilation device, characterized in that, The device includes: The acquisition module is used to acquire the upper-level motion intention, which is derived from at least one of natural language instructions, human-computer interaction interface, AI strategy model, visual language model, task planner or script interface; The first generation module is used to generate one or more candidate semantic frames based on the upper-layer motion intention. The candidate semantic frames contain semantic information of verbs, objects, modifiers, time and force parameters. The second generation module is used to standardize the candidate semantic frames, generate a standardized motion intention intermediate representation, and generate a unique hash value for the standardized motion intention intermediate representation. The standardization process includes unifying verbs, units, coordinate systems, object references, and control modes. The verification module performs multi-dimensional verification on the standardized motion intention intermediate representation. This multi-dimensional verification includes pattern verification, kinematic feasibility verification, control pattern compatibility verification, constraint conflict verification, and temporal feasibility verification. The constraint conflict verification involves dividing constraints into four levels based on priority: safety constraints, hard constraints, soft constraints, and preference constraints. Each level of constraint is checked sequentially to ensure it is satisfied. If a safety constraint or hard constraint is violated, the verification fails and a corresponding exception code is generated. If a soft constraint or preference constraint is violated, violation information is recorded and degraded execution is allowed. The control pattern compatibility verification involves constructing a control pattern compatibility matrix, which defines the compatibility relationships between position control, speed control, torque control, impedance control, and hybrid control modes. Based on the control pattern field in the standardized motion intention intermediate representation, the module queries the control pattern compatibility matrix to determine if the selected mode matches the current actuator capability and task stage. If the selected mode does not match the current actuator capability and task stage, a compatible alternative mode is automatically selected and the standardized motion intention intermediate representation is updated. The compilation module is used to construct a motion graph based on the standardized motion intention intermediate representation after verification, and compile the motion graph into low-level motion instructions. At the same time, it generates a compilation certificate containing the hash value, compilation result and model version. The construction of the motion graph based on the standardized motion intention intermediate representation includes: mapping each of the standardized motion intention intermediate representations to a node in the motion graph. The node attributes include target pose, control mode, expected force and time window. According to the temporal dependency and spatial interference conditions, directed edges are added between nodes. The directed edges carry the transition conditions or synchronization signals. The motion graph is topologically sorted to generate a sequence of instructions that can be executed sequentially or in parallel.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the robot control instruction compilation method as described in any one of claims 1 to 4.

7. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the robot control instruction compilation method as described in any one of claims 1 to 4.

Images