A robot cross-language high-frequency trajectory planning method and system based on an embedded tensor calculation engine

Abstract

The embodiment of the application discloses a robot cross-language high-frequency trajectory planning method and system based on an embedded tensor calculation engine. The scheme loads a packaging module through a bottom control terminal, and pulls up and initializes a script language runtime environment in the process space of the bottom control terminal. The memory direct access between the bottom control terminal and the runtime environment is realized based on a binding mechanism. High-frequency optimization solving can be realized between the bottom control terminal and the runtime environment in the runtime environment based on the current joint state information, target pose information and multi-dimensional soft constraint conditions acquired in real time. The target instruction is obtained by combining physical boundary hard clipping and is sent to a driver. In this way, the stability and safety of robot motion control can be improved through multi-dimensional soft constraint nonlinear optimization and motion limit control.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method and system for cross-language high-frequency trajectory planning for robots based on an embedded tensor computation engine. Background Technology

[0002] In the field of robot motion control, especially in the trajectory planning and joint control of multi-degree-of-freedom (e.g., 16-DOF) robots, the safety, stability, and global constraint capabilities of kinematic solution and path planning are the core requirements for ensuring the reliable operation of robots. They are directly related to the safety of the robot's own mechanical structure, the safety of the working environment, and the effectiveness of control commands.

[0003] Currently, the core functions of pure C++ kinematics solvers widely used in the industry are concentrated on calculating the forward and inverse kinematics solutions between joint space and Cartesian space. These solvers can usually only achieve inverse kinematics solutions within a local range. In complex work scenarios, when the robot's motion trajectory involves multi-joint coordination, numerous spatial obstacles, or complex target postures, relying solely on local inverse kinematics solutions can easily lead to collisions between the joint motion trajectory and the robot's own structure and the working environment, causing damage to the robot's mechanical components or work failure. Summary of the Invention

[0004] The purpose of this invention is to propose a cross-language high-frequency trajectory planning method for robots, which aims to solve the problems of insufficient stability and safety in existing robot motion control.

[0005] This invention provides a method for cross-language high-frequency trajectory planning for robots, the method comprising: The encapsulated module is loaded through the underlying control terminal, and the script language runtime environment is launched and initialized within the process space of the underlying control terminal. This is achieved through a binding mechanism, enabling direct memory access between the underlying control unit and the runtime environment. Within the runtime environment, high-frequency optimization is performed based on real-time acquired current joint state information, target pose information, and multi-dimensional soft constraint conditions. The target instruction is obtained by combining physical boundary hard clipping and then sent to the driver.

[0006] Accordingly, the present invention also provides a robot cross-language high-frequency trajectory planning device, comprising: The loading unit is used to load the encapsulated module through the underlying control terminal, and to start and initialize the script language runtime environment within the process space of the underlying control terminal. The access unit is used to enable direct memory access between the underlying control terminal and the runtime environment based on the binding mechanism. The solver unit is used to perform high-frequency optimization solutions based on real-time acquired current joint state information, target pose information, and multi-dimensional soft constraint conditions within the runtime environment. The dispatch unit is used to obtain the target instruction by combining physical boundary hard clipping and dispatch it to the driver.

[0007] Accordingly, the present invention also provides a robot cross-language high-frequency trajectory planning system, the entire system being deployed within a single process memory space, comprising: The underlying real-time control module runs in a real-time thread and is used to acquire the current physical state data of the robot's underlying hardware at high frequency, and send the target control commands after smooth interpolation to the underlying bus. The embedded algorithm calculation module runs in a non-real-time thread and includes a scripting language runtime environment and a tensor calculation engine, which is used to perform multidimensional soft-constraint nonlinear trajectory optimization. The cross-language bridging module is loaded and launched by the underlying real-time control module within a single process space, creating a scripting language runtime environment. The cross-language bridging module has a lock-free data interaction mechanism built inside, which is used to push the physical state data obtained by the underlying real-time control module into the embedded algorithm calculation module through direct memory mapping, and to send the target control instructions generated by the calculation back to the underlying real-time control module in the same way.

[0008] Accordingly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the above-described robot cross-language high-frequency trajectory planning method.

[0009] Accordingly, the present invention also provides a computer storage medium storing a plurality of instructions adapted for loading by a processor to execute the above-described robot cross-language high-frequency trajectory planning method.

[0010] This invention loads and encapsulates modules at the lower-level control terminal, launching and initializing a scripting language runtime environment within the lower-level control terminal's process space. Based on a binding mechanism, direct memory access between the lower-level control terminal and the runtime environment is achieved. Within the runtime environment, high-frequency optimization is performed based on real-time acquired current joint state information, target pose information, and multi-dimensional soft constraints. Combined with physical boundary hard trimming, target instructions are obtained and sent to the driver. Thus, through multi-dimensional soft-constraint nonlinear optimization and motion limit control, the stability and safety of robot motion control can be improved. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] in: Figure 1 This is a schematic diagram of a robot cross-language high-frequency trajectory planning system architecture provided in an embodiment of the present invention.

[0013] Figure 2 This is a flowchart illustrating a cross-language high-frequency trajectory planning method for robots, provided as an embodiment of the present invention.

[0014] Figure 3 This is a flowchart illustrating another cross-language high-frequency trajectory planning method for robots provided in an embodiment of the present invention.

[0015] Figure 4 This is a structural block diagram of a robot cross-language high-frequency trajectory planning device provided in an embodiment of the present invention.

[0016] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] This invention provides a method and system for cross-language high-frequency trajectory planning of robots based on an embedded tensor computation engine. Specifically, the cross-language high-frequency trajectory planning method of this invention can be executed by an electronic device, which can be a terminal or a server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms.

[0019] To address the aforementioned issues, this invention provides a robot cross-language high-frequency trajectory planning method and system based on an embedded tensor computation engine. Through multidimensional soft-constraint nonlinear optimization and motion limit control, it can improve the stability and safety of robot motion control.

[0020] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the preferred order of the embodiments.

[0021] In the field of control for complex humanoid or dual-arm robots, the underlying high-frequency hard real-time control (such as microsecond-level communication of the EtherCAT motor bus and hardware interpolation algorithms) must rely on a high-efficiency C++ environment. However, existing high-order nonlinear kinematics calculations and advanced trajectory planning (such as those based on JAX engines, deep learning, or cutting-edge optimization algorithm libraries like PyRoki) typically run natively and efficiently only within the Python ecosystem.

[0022] In related technologies, a "computation-separated cross-process architecture" is typically adopted. This means that within the operating system, the Python algorithm layer is treated as a separate server process, while the C++ control layer is treated as a separate client process. Data exchange between the two occurs through cross-process communication (IPC, Industrial Personal Computer) network mechanisms such as ROS 2 (Robot Operating System 2) Topics / Services, WebSockets, or RPC. However, cross-language architectures can introduce problems such as communication latency and delayed synchronization of asynchronous states.

[0023] First, the technical terms involved in this invention will be introduced as follows: JAX: A library developed by Google for high-performance numerical computing and machine learning, supporting AOT (Ahead-of-Time) compilation and GPU (Graphics Processing Unit) acceleration, and used in this invention to provide a low-level extreme computing engine.

[0024] JaxLS: A JAX-based nonlinear least squares solver for handling extremely complex kinematic optimization problems.

[0025] PyRoki: A cutting-edge robot kinematics and trajectory planning algorithm library based on the JAX engine, responsible for the trajectory planning verification of inverse kinematics (IK), forward kinematics (FK), and obstacle avoidance self-locking in this system.

[0026] PyBind11: A lightweight C++ library for seamless operation between C++ and Python. This invention utilizes it to implement PyrokiWrapper, deeply embedding the Python runtime into a ROS 2 C++ node to achieve zero-copy cross-language calls.

[0027] AOT: Ahead-of-Time (AOT) technology. It refers to pre-compiling the robot's URDF (Unified Robot Description Format, also known as the robot's "digital ID card + full-body structure diagram") tree-like joint and soft constraint diagrams, converting the dynamic computation graph into low-level high-efficiency machine code, thereby significantly reducing the time spent on inverse kinematics and trajectory generation.

[0028] IK: Inverse Kinematics. It refers to the algorithmic process of calculating the required angles of each joint from a given target pose (position and orientation) of the robot's end effector.

[0029] IPC (Inter-Process Communication): This refers to mechanisms for transmitting and exchanging data between different processes (such as Sockets and ROS 2 Topic / Service in traditional architectures). This invention aims to eliminate the communication latency caused by traditional IPC.

[0030] Soft constraints are mechanisms used in nonlinear optimization algorithms to limit system behavior by introducing a penalty cost. This invention includes higher-order difference penalties (suppressing acceleration spikes), self-collision penalties (avoiding limb crossing), and singularity removal penalties.

[0031] Hard Clipping: An absolute physical protection network set up at the underlying output. It refers to forcibly clipping the value within the safe physical limits ([lower, upper]) defined by URDF before the algorithm outputs the joint angle, using methods such as np.clip, so as not to issue any abnormal commands.

[0032] 16-DOF (16 Degrees of Freedom): 16 degrees of freedom. In this system, it specifically refers to the 16 active control motor axes that participate in the core derivative matrix calculation of the JAX trajectory planner. The specific physical distribution is: 7 axes for the left arm + 7 axes for the right arm + 2 axes for pitch and yaw.

[0033] Ruckig: A third-party open-source C++ algorithm library for high-speed motion planning and trajectory interpolation. In this system, it is responsible for receiving high-dimensional arrays generated by Python and performing secondary physical hardware clipping and high-speed three-dimensional smooth interpolation at the underlying level.

[0034] EtherCAT: A real-time industrial fieldbus communication protocol based on Ethernet, used in this system to achieve microsecond-level (hard real-time) high-frequency control communication for the underlying joint motors.

[0035] This application provides a robot cross-language high-frequency trajectory planning system. The system is deployed entirely within a single process memory space and includes: The underlying real-time control module runs in a real-time thread and is used to acquire the current physical state data of the robot's underlying hardware at high frequency, and send the target control commands after smooth interpolation to the underlying bus. The embedded algorithm calculation module runs in a non-real-time thread and includes a scripting language runtime environment and a tensor calculation engine, which is used to perform multidimensional soft-constraint nonlinear trajectory optimization. The cross-language bridging module is loaded and launched within a single process space by the underlying real-time control module, which also starts the scripting language runtime environment. Internally, the cross-language bridging module incorporates a lock-free data interaction mechanism. This mechanism directly maps physical state data acquired by the underlying real-time control module into the embedded algorithm calculation module, and then sends the calculated target control instructions back to the underlying real-time control module in the same manner. For example, please see Figure 1 , Figure 1 This is a schematic diagram of a robot cross-language high-frequency trajectory planning system architecture provided in an embodiment of the present invention.

[0036] The robot operating system can be ROS 2.

[0037] The underlying real-time control terminal (i.e., the underlying real-time control module) can be a C++ terminal.

[0038] The kinematics steward module, also known as the global singleton scheduling core, is responsible for loading the Python virtual machine when the node starts and managing cross-language data exchange.

[0039] In some embodiments, the kinematics manager module can be KinematicManage. KinematicManager is the core kinematic component of a high-degree-of-freedom robot cross-language high-frequency trajectory planning system. It is responsible for model management, unified calculation of FK / IK / Jacobian matrix, state caching and zero-latency synchronization, constraint safety checks and cross-language interface scheduling. It is the basic hub for realizing Cartesian control, real-time optimization and physical safety.

[0040] The cross-language bridging module functions as follows: it utilizes a C++ interface layer encapsulated in the PyBind11 library, deeply nested within the main thread, to directly map C++ Eigen matrices (mathematical arrays in C++ used for real-time control, EtherCAT, and kinematics) to Python-readable NumPy tensors (mathematical arrays in Python used for PyRoki, JAX, and optimization algorithms) using a lock-free mechanism (std::atomic), completely eliminating the serialization latency of traditional IPC (such as Socket and ROS Topic).

[0041] In some embodiments, the cross-language bridging module can be PyrokiWrapper, a cross-language wrapper that connects the C++ real-time control kernel with the Python PyRoki optimization library, thereby achieving zero copy, zero IPC latency, and bidirectional calls.

[0042] The high-speed interpolation module embeds the Ruckig algorithm library, which is responsible for receiving coarse-grained discrete waypoints generated by the Python side, performing cubic polynomial smooth interpolation and second-order (velocity / acceleration) physical limiting in millisecond-level interrupts, and finally sending them to the EtherCAT bus.

[0043] The underlying driver is implemented through the EtherCAT bus.

[0044] The web front-end is used to enable user interaction. It can provide a real-time remote control user interface on the web, allowing users to input control commands for the robot.

[0045] The embedded algorithm calculation end (i.e., the embedded algorithm calculation module) can be a Python version.

[0046] The asynchronous communication module is implicitly started in a non-real-time thread of the C++ process, listens on a specified web port (such as 8080), and is responsible for receiving asynchronous control JSON commands from the front-end UI (user interface).

[0047] In some embodiments, the asynchronous communication module can be server.py, which is specifically responsible for receiving external instructions, sending them to the robot's brain, and forwarding them to C++ for execution.

[0048] The core computing engine's role is to create a nonlinear least squares optimization graph based on Google JAX and the JaxLS library. It is responsible for parsing the robot's URDF model and generating a high-speed derivative calculation graph using AOT (Ahead-of-Time) technology.

[0049] In some embodiments, the core computing engine can be Kinematic.py, a custom kinematics utility class file used in ROS, which is generally used to: implement FK, IK, calculate Jacobian, perform joint angles, and transform end-effector poses, etc.

[0050] In some embodiments, the parameter configuration module is used to configure the physical parameters of the robot's URDF, as well as soft constraint configuration, and outputs the parameter configuration file Kinematic.yaml for use by the core algorithm engine.

[0051] This embodiment decouples the robot's physical boundaries and soft constraint penalty weights independently into the parameter configuration file (kinematic.yaml) and the URDF model. Through parameter decoupling, the system can quickly adapt to robot hardware with different configurations without modifying the core source code, thus improving engineering reusability.

[0052] For further details, please refer to Figure 2 , Figure 2 This is a flowchart illustrating a cross-language high-frequency trajectory planning method for robots, provided as an embodiment of the present invention. The specific flow of this cross-language high-frequency trajectory planning method for robots can be as follows: 101. Load the encapsulated module through the underlying control terminal, and start and initialize the script language runtime environment within the process space of the underlying control terminal.

[0053] The underlying control end, i.e., the C++ end, can be scripted in Python, and the runtime environment is a Python / JAX environment.

[0054] The encapsulation module is also known as the PyrokiWrapper encapsulation module.

[0055] For example, the C++ master node is used as the main process to start and initialize the system runtime environment; the PyrokiWrapper wrapper module is loaded and instantiated through the C++ master node; the PyrokiWrapper wrapper module is used to start and initialize the Python scripting language runtime environment inside the C++ main process, thus realizing the embedding of the Python runtime environment within the C++ process.

[0056] 102. Implement direct memory access between the underlying control terminal and the runtime environment based on the binding mechanism.

[0057] The binding mechanism can be the PyBind11 binding mechanism.

[0058] For example, based on the PyBind11 binding mechanism, cross-language direct memory access between the C++ master node and the Python environment is achieved without data serialization and intermediate file transfer; based on the direct access mechanism, robot physical state data, joint control data and safety constraint data are transmitted in real time between C++ modules and Python modules, completing cross-language collaborative control.

[0059] 103. Within the runtime environment, high-frequency optimization is performed based on the real-time acquired current joint state information, target pose information, and multi-dimensional soft constraint conditions.

[0060] The current joint state information refers to the current state information of each joint of the robot, which can include at least the joint angle parameters.

[0061] In some embodiments, the current joint state information may include the state information of the joints of the robot's upper body.

[0062] For example, obtaining the current joint state information of a robot can include: reading the robot's current 16-dimensional physical state matrix from the actual motor via a C++ interface to obtain the current joint state information. Here, the actual motor refers to the physical motor, which is the hardware actuator actually mounted on the robot joint, possessing physical windings, rotors, encoders, and drive circuits.

[0063] Specifically, the 16-dimensional physical state matrix refers to the state information of 16 joints, including: head pitch axis, head yaw axis, and so on. Yaw, left shoulder pitch (L_Shoulder_Pitch), left shoulder roll (L_Shoulder_Roll), left shoulder yaw (L_Shoulder_Yaw), left elbow pitch (L_Elbow_Pitch), left wrist yaw (L_Wrist_Yaw), left wrist pitch (L_Wrist_Pitch), left wrist roll (L_Wrist_Roll), right shoulder pitch (R_Shoulder_Pitch), right shoulder roll (R_Shoulder_Roll), right shoulder yaw (R_Shoulder_Yaw), right elbow pitch (R_Elbow_Pitch), right wrist yaw (R_Wrist_Yaw), right wrist pitch (R_Wrist_Pitch), right wrist roll (R_Wrist_Roll).

[0064] By applying a "waist-locking" constraint to the robot's upper body model (i.e., isolating the waist joint from the upper limb planning matrix), the rotation of the robot's waist joint will not change the relative coordinates of the two arms based on the upper body base, effectively reducing coordinate system coupling interference during hand-to-hand collaborative operations (such as precision assembly).

[0065] Among them, the target pose information is the target spatial parameters (X, Y, Z coordinates and Roll, Pitch, Yaw Euler angles) input by the user through the real-time remote operation user interface on the WEB terminal. The target pose information is used to characterize the endpoint constraints of the robot's motion.

[0066] In some embodiments, to improve the stability and safety of robot motion control, the step "performing high-frequency optimization based on real-time acquired current joint state information, target pose information, and multi-dimensional soft constraint conditions" may include the following operations: Based on the current joint state information and target pose information, an optimization objective function is constructed. Multidimensional soft constraints are introduced to modify the objective function of the optimization. An optimization algorithm is used to solve the modified objective function to obtain joint control information that satisfies the multidimensional soft constraint conditions.

[0067] The optimization objective function takes minimizing the deviation between the actual pose and the target pose of the robot's end effector as its core objective.

[0068] Specifically, the state deviation between the current joint state information and the target pose information is used as the basic optimization term to constrain the robot's end effector to converge toward the target pose. Secondly, multi-dimensional soft constraints are introduced, and the state deviation and multi-dimensional soft constraints are weighted and superimposed to form a comprehensive optimization objective function. The joint angle vector that minimizes the value of this optimization objective function is solved iteratively by a numerical optimization algorithm. This is the optimal joint state that satisfies the target pose and fits the current joint state, and is used as the joint control information.

[0069] The joint control information may include control parameters for each joint, such as angle parameters.

[0070] In some embodiments, multidimensional soft constraints may include at least: smoothing constraints and safety constraints; The step "Introducing multidimensional soft constraints to modify the objective function" can include the following operations: Smoothing constraints and safety constraints are introduced to jointly modify the objective function of optimization.

[0071] For example, by introducing smoothing constraints and safety constraints, and by weighting and superimposing the state deviation, smoothing constraints and safety constraints, a comprehensive optimization objective function can be formed.

[0072] In some embodiments, the smoothing constraints include at least the velocity constraints; the safety constraints include at least the collision constraints and the maneuverability constraints.

[0073] Among them, the velocity constraint is used to suppress acceleration spikes. For example, the velocity constraint can be: five_point_velocity_cost, which is the velocity smoothness cost term generated based on the fifth-order polynomial trajectory in robot trajectory optimization. It is used to penalize abrupt velocity changes and ensure continuous and smooth joint / end-effector velocities.

[0074] Collision constraints are used to prevent limbs from intersecting. For example, a collision constraint could be `self_collision_cost`, a penalty term constructed during robot trajectory optimization to prevent interference between the robot's links, joints, and end effectors. By monitoring the minimum distance between the robot's moving parts, the cost function value is automatically increased when the distance between parts is less than a safety threshold. This eliminates joint states prone to self-collision during the optimization process, ensuring interference-free and safe motion trajectories.

[0075] Among them, the operability constraint is used to expel singularity lock-up.

[0076] For example, the manipulability constraint can be manipulability_cost, used to improve the manipulability performance of the robot configuration during trajectory optimization, avoid motion entering singularities or weakly manipulable postures, and ensure that the end effector has sufficient force / position adjustment capability and motion flexibility. The lower the manipulability, the higher the cost; the higher the manipulability, the lower the cost.

[0077] In some embodiments, the step "constructing an optimization objective function based on the current joint state information and the target pose information" may include the following operations: The current joint state information is sliced ​​and reconstructed to obtain the joint state input data of the current joint state information under preset multiple degrees of freedom; Based on the joint state input data and target pose information, an optimization objective function is constructed.

[0078] Among them, the preset multiple degrees of freedom can include 16 degrees of freedom.

[0079] Specifically, the current joint state information is sliced ​​and reconstructed to obtain the joint state input data under a preset multi-degree-of-freedom condition. This may include: slicing and reconstructing the 16-dimensional physical state matrix into a standard JAX input matrix of 16-DOF (degrees of freedom) (including 2 axes for the head, 7 axes for the left arm, and 7 axes for the right arm), which is the joint state input data.

[0080] Furthermore, this input matrix is ​​injected into the computation graph as initial_vals (initial data), and used with the target pose information to construct the optimization objective function. The computation graph is also the optimization algorithm constructed by JAX, which essentially switches the current joint information to a data mode that JAX can process.

[0081] In some embodiments, to avoid hardware start-up jumps, the method may further include the following steps: Assign a preset static cost coefficient to the joint state input data.

[0082] The preset rest cost coefficient can be rest_cost, which is the penalty coefficient for keeping the robot in a static / natural posture. The larger the value (such as a maximum value like 10 to the power of 5), the heavier the penalty. The optimizer will try its best to prevent the robot from deviating from the static posture, that is, it forces the robot to stay as still as possible and maintain the initial posture.

[0083] In some embodiments, to enable real-time user control of the robot, the method may further include the following steps: Receive action commands for the robot; Obtain the robot's current joint state information, as well as the target pose information corresponding to the action command.

[0084] Among them, the action command instructs the robot to perform the corresponding action.

[0085] In some embodiments, the motion command may be triggered by a user. The step "Receive motion command for robot" may include the following operations: Obtain the target pose information input by the user through the operation interface; Action commands for the robot are triggered based on the target pose information.

[0086] The operation interface, also known as the real-time remote operation user interface on the web, may include a Cartesian control area, which can be used by users to plan the robot's motion trajectory.

[0087] For example, users can select a specific control arm (Left Arm or Right Arm) in the Cartesian space trajectory planning panel and input the target's spatial parameters (which may include: X, Y, Z coordinates and Roll, Pitch, Yaw Euler angles). After clicking "Execute," the front end pushes a JSON data packet to the server.py server, which is implicitly started by C++, via the WebSocket protocol. The user interface can then monitor the underlying feedback status in real time.

[0088] In some embodiments, the web-based real-time teleoperation user interface may also include a joint control area, which can be used by the user to independently, in real-time, and locally control individual joints of the robot.

[0089] In some embodiments, the method of receiving action instructions is not limited to user input, but may also include: directly feeding the action translation output of the robot's visual recognition system (such as 3D visual grasping pose estimation), large language model (LLM), or the output of reinforcement learning (RL) policy network as the target Cartesian coordinates to the server.py or KinematicManager of the present invention through internal calls.

[0090] In some embodiments, to ensure that subsequent calls are in a state of rapid readiness before receiving an action instruction, the following steps may also be performed: The system warms up and performs AOT compilation preprocessing. Upon system power-up, the C++ node starts, PyrokiWrapper launches the Python environment, and the JAX engine is initialized. The system parses the kinematic.yaml configuration file and the URDF physical mesh, completing the construction of the collision bounding box and JIT preprocessing to ensure subsequent calls are in a state of near-instantaneous readiness.

[0091] Among them, the power-on of the AOT compilation preprocessing system refers to the loading, verification, memory mapping and initialization of the pre-compiled (AOT) system / application code during the power-on phase of the device, so that the system can directly execute machine code and start up quickly.

[0092] 104. Combine physical boundary hard clipping to obtain the target instruction and send it to the driver.

[0093] In some embodiments, to avoid the solved joint control information from exceeding the joint movement limit range and causing safety issues, the step "obtaining the target instruction by combining physical boundary hard trimming" may include the following operations: Obtain the joint movement limit range of the robot; Based on the motion limit range of each joint in the joint motion limit range, the control parameters of each joint in the joint control information are trimmed and constrained to obtain target joint control information that does not exceed the joint motion limit range. The target command is generated based on the target joint control information.

[0094] The joint motion limit range refers to the absolute physical boundaries of the joint's range of motion [lower, upper]. Lower represents the minimum position / angle the joint can reach, and upper represents the maximum position / angle the joint can reach. The joint motion limit range provides insurmountable hard constraints for trajectory planning, motion optimization, collision detection, and control, ensuring that joint motion always remains within the safe range allowed by the mechanical design, preventing structural damage, actuator overload, or system failure due to overtravel.

[0095] Specifically, the joint motion limit range includes the [lower, upper] corresponding to each joint of the robot. Based on the [lower, upper] corresponding to the joint, the control parameters of each joint in the joint control information that exceed the [lower, upper] are clipped so that the control parameters of each joint in the joint control information are all within the corresponding [lower, upper].

[0096] In some embodiments, to ensure the smoothness of robot movement, the method may further include the following operations: Interpolation is performed based on the current joint state information and the target joint control information to generate continuously transitioning target joint control information.

[0097] For example, based on the robot's current joint state and target pose information, multiple (e.g., 50) high-frequency safe interpolation points are generated at equal intervals or adaptively along the motion path as discrete constraint nodes for trajectory tracking and optimization. These interpolation points cover the complete motion range from the initial pose to the target pose, ensuring smooth and continuous trajectory through dense sampling. Each interpolation point contains joint position and velocity constraint information at the corresponding moment, and simultaneously performs self-collision detection, joint limit verification, and operability assessment, eliminating unsafe pose points with interference, exceeding limits, or singular risks. Based on this set of high-frequency safe interpolation points, a continuous and smooth joint motion trajectory is constructed, ensuring that the robot remains in a safe, stable, and collision-free motion range throughout the entire motion process, while simultaneously meeting the requirements for end-effector pose tracking accuracy and motion smoothness.

[0098] This scheme introduces multi-dimensional soft constraint cost matrices into the JAX computation graph, including a smoothing penalty utilizing central high-order differences (to suppress acceleration spikes), an interference penalty based on URDF collision bounding boxes (to avoid self-interference), and a Jacobi operability penalty (to drive away singularities). These high-dimensional computational parameters are completely decoupled from the robot's physical boundaries into independent configuration files, and AOT (Ahead-of-Time) technology is used for extremely rapid differentiation. This endows the trajectory planning with compliant obstacle avoidance capabilities while ensuring high-frequency convergence of the algorithm.

[0099] In some embodiments, the step "generating the target command based on the target joint control information" may include the following operations: The target joint control information is pushed directly from memory to the underlying control terminal through the runtime environment. The target joint control information is converted into motion control commands through the underlying control terminal.

[0100] Specifically, the runtime environment pushes the valid target joint control information (two-dimensional tensor) directly from memory to the `trajectory_msgs / JointTrajectory` structure on the C++ side. The `TrajectoryPlanner` on the C++ side takes over the data, performs Ruckig interpolation filtering, and finally converts it into smooth pulse commands, which are then sent to the motor driver via EtherCAT to drive the motor to complete the corresponding action.

[0101] In some embodiments, in order to facilitate users to observe the robot's motion status in real time, the web-based real-time teleoperation user interface can present the monitored underlying feedback status. Finally, users can observe on the user interface that the robot smoothly reaches the target pose according to the motion commands.

[0102] This invention discloses a cross-language high-frequency trajectory planning method for robots. The method includes: receiving motion commands for the robot; acquiring the robot's current joint state information; performing optimization based on the current joint state information, the target pose information corresponding to the motion command, and multi-dimensional soft constraints to obtain joint control information; processing the joint control information according to the robot's corresponding joint motion limit range to obtain target joint control information; and controlling the robot to execute the motion command based on the target joint control information. Thus, through multi-dimensional soft-constraint nonlinear optimization and motion limit control, the stability and safety of robot motion control can be improved.

[0103] Based on the above description, the following examples will further illustrate the robot cross-language high-frequency trajectory planning method of the present invention. Please refer to... Figure 3 , Figure 3The flowchart of another robot cross-language high-frequency trajectory planning method provided in this embodiment of the invention is shown below: Receive target Cartesian pose: Specifically, the system receives a Cartesian space movement command for the robot's end effector.

[0104] The movement command specifies the target position and orientation of the robot's end effector in Cartesian space. It instructs the robot to plan the end effector's motion path in Cartesian space based on the movement command, and convert it into the motion trajectory in the corresponding joint space through kinematic solution, so that the robot's end effector can accurately approach the target orientation along a Cartesian straight line path or a smooth curve path.

[0105] Panoramic physical state extraction and multi-degree-of-freedom anchoring: The robot's real-world joint data is extracted from the underlying layers, sliced ​​and reconstructed into a standard 16-DOF (degrees of freedom) JAX input matrix, injected into the first frame of the computation graph, and assigned... The rest_cost, which is the first layer of lock, is used to prevent the robot from jittering when starting.

[0106] In some embodiments, the scheme with the first locking can also be implemented by replacing the nonlinear least squares optimizer (LS) with a sequential quadratic programming solver (SQP) or interior-point method solver (IPM) that supports hard constraints, instead of using extremely large soft-penalty weights. During optimization modeling, the trajectory's... The frame is strictly bound to the actual physical angle being read through an "Equality Constraint".

[0107] Nonlinear least squares optimization: JAX begins iterative processing of the dynamic computation graph compiled using AOT. Specifically, the JAX engine iteratively optimizes the target pose (i.e., the position and orientation information carried in the movement command, which is input by the user).

[0108] In solving the gradient, the algorithm simultaneously calculates the penalty terms of multiple soft constraints: smoothness constraint: using five_point_velocity_cost to penalize velocity mutations; safety constraint: calculating self_collision_cost to avoid trunk collisions, and calculating manipulability_cost to drive away singularities of the robotic arm.

[0109] Physical hard cutting: The target joint array obtained in the previous step is used to call the np.clip method to strictly bind all tensors within the [lower, upper] physical upper limit defined by URDF, that is, to add a second lock to prevent out-of-bounds roll-off.

[0110] Lock-free cross-end delivery and secondary smoothing: Approximately 50 high-frequency secure interpolation points are generated. These secure interpolation points are then pushed into a C++ TrajectoryPlanner for millisecond-level continuous time difference and second-order hardware limiting.

[0111] In some embodiments, the underlying hardware limiting and smooth interpolation can also be implemented in the following ways: using the Reflexxes Motion Library (Type II / Type IV), or writing a custom interpolation algorithm based on quintic spline interpolation, B-spline curves, or Bezier curves, and combining it with the limiting filter of the underlying motor driver itself to achieve the same function of "receiving discrete safe waypoints, generating continuous smooth pulses, and limiting them".

[0112] Smoothly send control commands to the driver: The data is sent to the motor driver via EtherCAT, which then drives the physical robot to complete the action.

[0113] This embodiment proposes a dual insurance mechanism of "input-side mathematical deadlock" and "output-side physical clipping". In the first step of optimization, the current 16 degrees of freedom real physical angle captured by the underlying layer is forcibly used as an equality constraint (injecting a huge penalty cost) for absolute state anchoring; and an np.clip physical wall is set up at the algorithm output to perform secondary out-of-bounds filtering (i.e., hard clipping). This can greatly avoid joint rollback, start-up jitter and motor "runaway" phenomena that may occur when the robot receives high-frequency extreme illegal coordinates, thus achieving industrial-grade control safety.

[0114] To facilitate better implementation of the robot cross-language high-frequency trajectory planning method provided in the embodiments of the present invention, this application also provides a robot cross-language high-frequency trajectory planning device based on the above-described robot cross-language high-frequency trajectory planning method. The meanings of the terms used are the same as in the above-described robot cross-language high-frequency trajectory planning method, and specific implementation details can be found in the descriptions in the method embodiments.

[0115] Please see Figure 4 , Figure 4 This invention provides a structural block diagram of a robot cross-language high-frequency trajectory planning device, which includes: Loading unit 301 is used to load the encapsulated module through the underlying control terminal, and to start and initialize the script language runtime environment within the process space of the underlying control terminal. Access unit 302 is used to enable direct memory access between the underlying control terminal and the runtime environment based on a binding mechanism; Solver 303 is used to perform high-frequency optimization solutions based on real-time acquired current joint state information, target pose information and multi-dimensional soft constraint conditions within the runtime environment. The sending unit 304 is used to obtain the target instruction by combining the physical boundary hard clipping and send it to the driver.

[0116] In some embodiments, the solving unit 303 may include: Construct sub-units to build an optimization objective function based on the current joint state information and the target pose information. The modified sub-unit is used to introduce multidimensional soft constraints to modify the optimization objective function; The sub-unit is used to solve the modified objective function using an optimization algorithm to obtain joint control information that satisfies the multidimensional soft constraint conditions.

[0117] In some embodiments, multidimensional soft constraints include at least: smoothing constraints and safety constraints; The modified subunit may specifically include: Smoothing constraints and safety constraints are introduced to jointly modify the objective function of optimization.

[0118] In some embodiments, constructing a subunit may specifically include: The current joint state information is sliced ​​and reconstructed to obtain the joint state input data of the current joint state information under preset multiple degrees of freedom; Based on the joint state input data and target pose information, an optimization objective function is constructed.

[0119] In some embodiments, the device may further include: The assignment unit is used to assign a preset static cost coefficient to the joint state input data.

[0120] In some embodiments, the sending unit 304 may include: Acquire sub-units to obtain the joint motion limit range of the robot; The processing subunit is used to trim and constrain the control parameters of each joint in the joint control information according to the motion limit range of each joint in the joint motion limit range, so as to obtain the target joint control information that does not exceed the joint motion limit range. The generation subunit is used to generate target instructions based on the target joint control information.

[0121] In some embodiments, the generating subunit can be specifically used for: The target joint control information is pushed directly from memory to the underlying control terminal through the runtime environment. The target joint control information is converted into motion control commands through the underlying control terminal.

[0122] In some embodiments, the device may further include: The generation unit is used to perform interpolation processing based on the current joint state information and the target joint control information to generate continuously transitioning target joint control information.

[0123] In some embodiments, the device may further include: The receiving unit is used to receive motion commands for the robot; The acquisition unit is used to acquire the current joint state information of the robot and the target pose information corresponding to the action command.

[0124] In some embodiments, the receiving unit may include: The second acquisition unit is used to acquire the target pose information input by the user through the operation interface; The triggering unit is used to trigger motion commands for the robot based on the target pose information.

[0125] This invention discloses a robot cross-language high-frequency trajectory planning device. A loading unit 301 loads an encapsulated module through a low-level control terminal, launching and initializing a scripting language runtime environment within the process space of the low-level control terminal. An access unit 302 enables direct memory access between the low-level control terminal and the runtime environment based on a binding mechanism. A solution unit 303 performs high-frequency optimization within the runtime environment based on real-time acquired current joint state information, target pose information, and multi-dimensional soft constraints. A distribution unit 304 combines physical boundary hard trimming to obtain the target instruction and distributes it to the driver. Thus, through multi-dimensional soft constraint nonlinear optimization and motion limit control, the stability and safety of robot motion control can be improved.

[0126] Accordingly, embodiments of the present invention also provide an electronic device. For example... Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 400 includes a processor 401 with one or more processing cores, a memory 402 with one or more computer storage media, and a computer program stored in the memory 402 and executable on the processor. The processor 401 and the memory 402 are electrically connected. Those skilled in the art will understand that... Figure 5The electronic device structures shown herein do not constitute a limitation on electronic devices and may include, but are not limited to, those described above. Figure 5 It can show more or fewer parts, or combine certain parts, or arrange different parts.

[0127] The processor 401 is the control center of the electronic device 400. It connects various parts of the electronic device 400 through various interfaces and lines. By running or loading software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, it performs various functions of the electronic device 400 and processes data, thereby monitoring the electronic device 400 as a whole.

[0128] In this embodiment of the invention, the processor 401 in the electronic device 400 loads the instructions corresponding to the processes of one or more application programs into the memory 402 according to the following steps, and the processor 401 runs the application programs stored in the memory 402 to realize various functions: The encapsulated module is loaded through the underlying control terminal, and the script language runtime environment is launched and initialized within the process space of the underlying control terminal. This is achieved through a binding mechanism, enabling direct memory access between the underlying control unit and the runtime environment. Within the runtime environment, high-frequency optimization is performed based on real-time acquired current joint state information, target pose information, and multi-dimensional soft constraint conditions. The target instruction is obtained by combining physical boundary hard clipping and then sent to the driver.

[0129] This invention, through a low-level control terminal loading an encapsulation module, launches and initializes a scripting language runtime environment within the process space of the low-level control terminal. A binding mechanism enables direct memory access between the low-level control terminal and the runtime environment. Within the runtime environment, high-frequency optimization is performed based on real-time acquired joint state information, target pose information, and multi-dimensional soft constraints. Target instructions are obtained by combining physical boundary hard trimming and then sent to the driver. Thus, through multi-dimensional soft constraint nonlinear optimization and motion limit control, the stability and safety of robot motion control can be improved.

[0130] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0131] Optional, such as Figure 5 As shown, the electronic device 400 may further include a display 403 and an input unit 404. The processor 401 is electrically connected to both the display 403 and the input unit 404. Those skilled in the art will understand that... Figure 5The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0132] Display 403 can be used to display a graphical user interface (GUI) and receive operation commands generated by the user interacting with the GUI. Display 403 may include a display panel and a touch panel. The display panel can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the electronic device. These graphical user interfaces can be composed of graphics, guidance information, icons, video, and any combination thereof. Optionally, the display panel can be configured using a liquid crystal display (LCD), organic light-emitting diode (OLED), or other similar technologies. The touch panel can be used to collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel), generate corresponding operation commands, and execute the corresponding program according to the operation commands. Optionally, the touch panel may include a touch detection device and a touch controller.

[0133] The touch detection device detects the user's touch location and the signal generated by the touch operation, transmitting the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 401. It can also receive and execute commands from the processor 401. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it transmits the information to the processor 401 to determine the type of touch event. Subsequently, the processor 401 provides corresponding visual output on the display panel based on the type of touch event. In this embodiment, the touch panel and display panel can be integrated into the display 403 to achieve input and output functions. However, in some embodiments, the touch panel and display panel can be implemented as two independent components to achieve input and output functions. That is, the display 403 can also be used as part of the input unit 404 to achieve input functions.

[0134] The input unit 404 can be used to receive input numbers, characters, or user characteristic information (such as fingerprints, iris, facial information, etc.), and generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.

[0135] In some embodiments, the electronic device may further include an audio circuit, which can provide an audio interface between the user and the device control device via a speaker and a microphone. The audio circuit can convert received audio data into electrical signals and transmit them to the speaker, where the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by the audio circuit, converted back into audio data, and processed by the processor 401. The audio data is then transmitted via a radio frequency circuit to, for example, another device control device, or output to a memory 402 for further processing. The audio circuit may also include an earphone jack to provide communication between a peripheral headset and the device control device.

[0136] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0137] As can be seen from the above, the electronic device provided in this embodiment can load the encapsulation module through the underlying control terminal, and start and initialize the script language runtime environment within the process space of the underlying control terminal; it can realize direct memory access between the underlying control terminal and the runtime environment based on the binding mechanism; within the runtime environment, it can perform high-frequency optimization based on the real-time acquired current joint state information, target pose information and multi-dimensional soft constraint conditions; and it can obtain the target instruction by combining physical boundary hard trimming and send it to the driver.

[0138] To this end, embodiments of the present invention provide a computer storage medium storing a plurality of computer programs, which can be loaded by a digital signal processor to execute the steps in any of the device control methods provided in the embodiments of the present invention. For example, the computer program can execute the following steps: The encapsulated module is loaded through the underlying control terminal, and the script language runtime environment is launched and initialized within the process space of the underlying control terminal. This is achieved through a binding mechanism, enabling direct memory access between the underlying control unit and the runtime environment. Within the runtime environment, high-frequency optimization is performed based on real-time acquired current joint state information, target pose information, and multi-dimensional soft constraint conditions. The target instruction is obtained by combining physical boundary hard clipping and then sent to the driver.

[0139] This invention, through a low-level control terminal loading an encapsulation module, launches and initializes a scripting language runtime environment within the process space of the low-level control terminal. A binding mechanism enables direct memory access between the low-level control terminal and the runtime environment. Within the runtime environment, high-frequency optimization is performed based on real-time acquired joint state information, target pose information, and multi-dimensional soft constraints. Target instructions are obtained by combining physical boundary hard trimming and then sent to the driver. Thus, through multi-dimensional soft constraint nonlinear optimization and motion limit control, the stability and safety of robot motion control can be improved.

[0140] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0141] The computer storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0142] Since the computer program stored in the computer storage medium can execute the steps in any of the device control methods provided in the embodiments of the present invention, the beneficial effects that any of the device control methods provided in the embodiments of the present invention can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0143] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for cross-language high-frequency trajectory planning for robots, characterized in that, The method includes: The encapsulation module is loaded through the underlying control terminal, and the script language runtime environment is launched and initialized within the process space of the underlying control terminal. This is achieved through a binding mechanism, enabling direct memory access between the underlying control unit and the runtime environment. Within the runtime environment, high-frequency optimization is performed based on real-time acquired current joint state information, target pose information, and multi-dimensional soft constraint conditions. The target instruction is obtained by combining physical boundary hard clipping and then sent to the driver.

2. The method according to claim 1, characterized in that, The high-frequency optimization solution based on real-time acquired current joint state information, target pose information, and multi-dimensional soft constraint conditions includes: Based on the current joint state information and target pose information, an optimization objective function is constructed. The multidimensional soft constraint conditions are introduced to modify the optimization objective function; An optimization algorithm is used to solve the modified objective function to obtain joint control information that satisfies the multidimensional soft constraint conditions.

3. The method according to claim 2, characterized in that, The multidimensional soft constraints include at least: smoothing constraints and safety constraints; The process of introducing the multidimensional soft constraints to modify the optimization objective function includes: The smoothing constraint and the safety constraint are introduced together to modify the objective function of optimization.

4. The method according to claim 2, characterized in that, The step of constructing an optimization objective function based on the current joint state information and the target pose information includes: The current joint state information is sliced ​​and reconstructed to obtain the joint state input data of the current joint state information under a preset multi-degree-of-freedom condition; Based on the joint state input data and the target pose information, the optimization objective function is constructed.

5. The method according to claim 4, characterized in that, The method further includes: A preset static cost coefficient is assigned to the joint state input data.

6. The method according to claim 1, characterized in that, The target instruction obtained by combining physical boundary hard clipping includes: Obtain the joint movement limit range of the robot; Based on the motion limit range of each joint in the joint motion limit range, the control parameters of each joint in the joint control information are trimmed and constrained to obtain target joint control information that does not exceed the joint motion limit range. The target command is generated based on the target joint control information.

7. The method according to claim 6, characterized in that, The method further includes: Interpolation is performed based on the current joint state information and the target joint control information to generate continuously transitioning target joint control information.

8. The method according to claim 7, characterized in that, The generation of the target command based on the target joint control information includes: The target joint control information is pushed directly from memory to the underlying control terminal through the runtime environment; The target joint control information is converted into motion control commands through the underlying control terminal.

9. The method according to claim 1, characterized in that, The method further includes: Receive action commands for the robot; Obtain the current joint state information of the robot, as well as the target pose information corresponding to the action command.

10. The method according to claim 9, characterized in that, The receiving of action commands for the robot includes: Obtain the target pose information input by the user through the operation interface; Action commands for the robot are triggered based on the target pose information.

11. A robot cross-language high-frequency trajectory planning device, characterized in that, The device includes: The loading unit is used to load the encapsulated module through the underlying control terminal, and to start and initialize the script language runtime environment within the process space of the underlying control terminal. The access unit is used to enable direct memory access between the underlying control terminal and the runtime environment based on the binding mechanism. The solution unit is used to perform high-frequency optimization solutions based on the real-time acquired current joint state information, target pose information and multi-dimensional soft constraint conditions within the runtime environment. The dispatch unit is used to obtain the target instruction by combining physical boundary hard clipping and dispatch it to the driver.

12. A robot cross-language high-frequency trajectory planning system, characterized in that, The system is deployed entirely within a single process memory space, including: The underlying real-time control module runs in a real-time thread and is used to acquire the current physical state data of the robot's underlying hardware at high frequency, and send the target control commands after smooth interpolation to the underlying bus. The embedded algorithm calculation module runs in a non-real-time thread and includes a scripting language runtime environment and a tensor calculation engine, which is used to perform multidimensional soft-constraint nonlinear trajectory optimization. The cross-language bridging module is loaded and launched by the underlying real-time control module within a single process space, which is a lock-free data interaction mechanism built inside the cross-language bridging module. This mechanism is used to push the physical state data obtained by the underlying real-time control module into the embedded algorithm calculation module through direct memory mapping, and to send the target control instructions generated by the calculation back to the underlying real-time control module in the same way.

13. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the robot cross-language high-frequency trajectory planning method as described in any one of claims 1 to 10.

14. A computer storage medium, characterized in that, The computer storage medium stores multiple instructions, which are adapted for loading by a processor to execute the robot cross-language high-frequency trajectory planning method according to any one of claims 1 to 10.

Images