Inverse kinematics solution and trajectory planning method for robotic arm based on digital twin technology

Abstract

The present invention discloses a method for solving inverse kinematics and trajectory planning of a robotic arm based on digital twin technology, including constructing a digital twin platform to achieve virtual-real interaction; specifically, building a virtual environment and establishing virtual-real interactive communication; conducting reinforcement learning training based on the SAC algorithm to construct an inverse solution model for the robotic arm; specifically, including state space design, action space design, gradually optimizing the inverse kinematics solution model through reward functions and curriculum learning strategies, model export, and model integration; using polynomial interpolation to plan the motion trajectory of the robotic arm and generate a smooth trajectory. The inverse kinematics solution method based on reinforcement learning in this method can quickly adapt to different target tasks, and achieve the optimal solution through efficient reward function design, which can solve the problems of long solution time and poor adaptability of traditional methods.

Description

Technical Field

[0001] The present invention relates to the field of intelligent manufacturing and robotics technology, and in particular to a method for solving inverse kinematics and trajectory planning of a robotic arm based on digital twin technology. Background Art

[0002] With the advancement of smart manufacturing and Industry 4.0, the application of industrial robots in automated production is becoming increasingly widespread. However, traditional robotic control methods face bottlenecks in accuracy, efficiency, and adaptability when dealing with complex environments and multi-tasking collaboration. As the core execution unit of industrial robots, solving inverse kinematics for the robotic arm is the foundation for achieving high-precision motion control.

[0003] Currently, common inverse kinematics solution methods include analytical methods and numerical iteration methods. Analytical methods obtain explicit solutions for joint angles through mathematical derivation, but their applicability is limited, and they can only solve problems with robotic arms with relatively simple geometric structures. Numerical iteration methods gradually approach the target solution through multiple calculations, and are suitable for complex structures. However, they suffer from the time-consuming real-time calculations and sensitivity to initial values, making them difficult to meet the requirements of real-time and high-precision control.

[0004] In recent years, the rise of digital twin technology has provided a new technical direction for robotic arm control. By constructing a digital model consistent with the physical entity in a virtual environment, digital twins can enable real-time monitoring, simulation, and prediction of the robotic arm. However, the application of digital twin technology remains a technical challenge in achieving rapid solution of the robotic arm's inverse kinematics and efficient trajectory planning. Summary of the Invention

[0005] In response to the above-mentioned technical deficiencies, the purpose of the present invention is to provide a method for solving inverse kinematics and trajectory planning of a robotic arm based on digital twin technology, which can solve the problems of long solution time and poor adaptability of traditional methods.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] The present invention provides a method for solving inverse kinematics and trajectory planning of a robotic arm based on digital twin technology, comprising the following steps:

[0008] S1. Build a digital twin platform to achieve virtual-reality interaction; specifically, it includes setting up a virtual environment and establishing virtual-reality interactive communication;

[0009] S2. Conduct reinforcement learning training based on the SAC algorithm to build an inverse kinematics solution model for the robotic arm. This includes state space design, action space design, gradual optimization of the inverse kinematics solution model through reward functions and curriculum learning strategies, model export, and model integration.

[0010] S3. Use polynomial interpolation to plan the robot arm motion trajectory and generate a smooth trajectory.

[0011] Preferably, in step S1, building the virtual environment specifically includes: using the URDF Importer component to import the URDF file of the physical robotic arm into Unity, ensuring that the structure, size, and joint layout of the digital robotic arm are completely consistent with the actual robotic arm;

[0012] Then, the physical parameters of the digital robotic arm model, including the range of motion, inertia characteristics, and friction coefficient of the joints, were added to simulate the dynamic characteristics of the actual robotic arm. A corresponding virtual environment was created based on the experimental requirements, including target points, obstacles, and a 3D scene for the experiment.

[0013] The specific steps to build virtual-reality interactive communication are: establishing a real-time communication interface between Unity and ROS through ROS-Connector and ROS-EndPoint to achieve state synchronization between the virtual environment and the actual robotic arm;

[0014] Configure ROS topics for transmitting joint angles θ, end-point poses (x, y, z, roll, pitch, yaw), and sensor data, and define message formats to ensure efficient and accurate real-time communication.

[0015] Finally, ensure that the operating status of the robotic arm can be transmitted to the virtual environment in real time, and at the same time return the feedback data of the virtual environment to the physical robotic arm.

[0016] Preferably, step S2 is specifically:

[0017] S21. State Space Design: Build a reinforcement learning environment based on the Unity platform's ML-Agents toolkit and define the state space of the robotic arm. The state space includes:

[0018] The current position of the end effector (x, y, z, roll, pitch, yaw) is used to characterize the control accuracy of the robotic arm; the current angle of each joint (θ1, ...θ n ), n depends on the degree of freedom of the manipulator; at the beginning of each cycle, a target position (x target ,y target ,z target ), as the core task goal of inverse kinematics solution;

[0019] S22, Action Space Design: Using the ContinuousAction mechanism of ML-Agents, the angles of each joint of the robot arm (θ1, ...θ n) As the output of the continuous action space, set the action range to (-180°, 180°) to ensure that the actions comply with the kinematic constraints of the joints;

[0020] S23. Gradually optimize the inverse kinematics solution model through the reward function and curriculum learning strategy: During the task execution, calculate the Euclidean distance d between the end effector and the target point in real time, and design the reward function based on the distance: The specific reward settings are as follows:

[0021]

[0022] The total reward is as described in formula (2)

[0023]

[0024] where, R t is the reward value at the t-th step, T is the number of cycles of execution, d0 and d1 respectively represent the minimum and maximum boundaries of the distance threshold, and d0 < d1 is satisfied; when d ≤ d1, the cycle ends;

[0025] Specifically

[0026] When d ≤ d0, it is considered that the distance between the robotic arm and the target position is small enough. At this time, R t = 100, indicating that the task has been successfully completed;

[0027] When d0 < d ≤ d1, the robotic arm is within the buffer range of the target position, and the reward value decreases linearly according to the change of ddd. Specifically to encourage the robotic arm to approach the target;

[0028] When d > d0, it is considered that the robotic arm is too far from the target, and the reward value is R <00000(8) = -0.01 to reflect that the task is not ideal and continue training.

[0029] In addition, introduce the curriculum learning strategy:

[0030] In the initial stage, set a larger target error range to reduce the task complexity and accelerate the model's learning of the basic control strategy; the larger target error range is d1 = 50mm, d0 = 20mm;

[0031] As the training progresses, gradually narrow the target range (d1 = 20mm, d0 = 10mm), increase the task difficulty and strengthen the model's fine control ability; narrow the target range d1 = 20mm, d0 = 10mm;

[0032] [[ID=!47]]In the final stage, narrow the target range to industrial-level accuracy to achieve high-precision inverse kinematics solution; the industrial-level accuracy is d1 = 5mm, d0 = 1mm;

[0033] S24. Model export: Use the model save and export function of ML-Agents to export the trained reinforcement learning model to ONNX format;

[0034] S25. Model integration: Integrate the exported ONNX model into Unity and load the model through the Barracuda plug-in.

[0035] Preferably, step S3 specifically includes:

[0036] S31, randomly generate target points (x target ,y target ,z target )Get the current joint angles (θ1, ...θ n ) and the end pose (x, y, z, roll, pitch, yaw) are passed into the ONNX model obtained in step 2 to obtain the inverse angle of each joint (θ1', ...θ n ');

[0037] S32, using a fifth-order polynomial to generate the trajectory curve to ensure the continuity and smoothness of the path;

[0038] S33. Import the interpolated trajectory into the digital twin platform and run the simulation on the digital robotic arm to verify the feasibility of the trajectory.

[0039] The beneficial effects of the present invention are:

[0040] (1) The inverse kinematics solution method based on reinforcement learning can quickly adapt to different target tasks and achieve the optimal solution through efficient reward function design.

[0041] (2) Introduce curriculum learning strategies to gradually improve the model's ability to solve problems in complex environments, making it applicable to complex environments.

[0042] (3) By training and verifying in a virtual environment, the development cost and debugging difficulty before actual deployment are greatly reduced.

[0043] (4) The model is exported in ONNX format and integrated into the Unity platform, ensuring the compatibility and efficiency of the system in various application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 A schematic diagram of a robotic arm model provided by an embodiment of the present invention;

[0046] Figure 2 A schematic diagram of a reinforcement learning trajectory planning method provided by an embodiment of the present invention;

[0047] Figure 3 The reward variation diagram under the target error range of d1 = 50mm and d0 = 20mm;

[0048] Figure 4 The reward variation diagram under the target error range of d1 = 20 mm and d0 = 10 mm is shown;

[0049] Figure 5 This is a graph showing the reward changes within the target error range of d1 = 5 mm and d0 = 1 mm. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0051] like Figures 1 to 3 As shown in FIG, a method for solving inverse kinematics and trajectory planning of a robotic arm based on digital twin technology includes the following steps:

[0052] Step 1: Use the 6-DOF robotic arm of Zhongke Shengu ROCR6 (see Figure 1 ), building a digital twin platform to achieve virtual-reality interaction; specifically:

[0053] S11. Build a virtual environment;

[0054] Use the URDF Importer to import the physical robotic arm's URDF file into Unity, ensuring that the digital robotic arm's structure, dimensions, and joint layout are identical to the actual robotic arm. Then, add its physical parameters, including joint range of motion, inertia, and friction coefficient, to the Unity digital robotic arm model to simulate the dynamic characteristics of the actual robotic arm. Create a corresponding virtual environment based on the experimental requirements. For simplicity, no obstacles are set up here.

[0055] S12. Build virtual-real interactive communication;

[0056] A real-time communication interface between Unity and ROS is established through ROS-Connector and ROS-EndPoint, synchronizing the virtual environment with the physical robotic arm. The robotic arm's / joint_state topic is configured to transmit joint angles, end-point pose, and sensor data, and a custom message format is defined for this topic to ensure efficient and accurate real-time communication. Finally, the robotic arm's operational status (such as joint angles, velocity, and end-point pose) is transmitted to the virtual environment in real time, while feedback from the virtual environment is returned to the physical robotic arm.

[0057] Step 2: Perform reinforcement learning training based on the SAC algorithm to build an inverse solution model for the robotic arm. This includes the following:

[0058] S21, state space design;

[0059] A reinforcement learning environment is built based on the ML-Agents toolkit on the Unity platform to define the state space of the robotic arm.

[0060] The state space consists of:

[0061] The current pose (x, y, z, roll, pitch, yaw) of the end effector of the digital entity is used to characterize the control accuracy of the robot arm; the current angle of each joint (θ1, θ2, θ3, θ4, θ5, θ6); at the beginning of each cycle, a random initial target position (x target ,y target ,z target ), as the core task goal of inverse kinematics solution.

[0062] S22, action space design;

[0063] Using the ContinuousAction mechanism of ML-Agents, the angles of each joint of the robotic arm (θ1, ...θ n ) as the output of the continuous action space, and set the action range to (-180°, 180°) to ensure that the action complies with the kinematic constraints of the joint;

[0064] S23, gradually optimize the inverse kinematics solution model through reward function and curriculum learning strategy;

[0065] During the task execution, the Euclidean distance d between the end effector and the target point is calculated in real time, and the reward function is designed based on the distance:

[0066] In the present invention, the reward is set as follows:

[0067]

[0068] The total reward is as described in Equation (2).

[0069]

[0070] Where R t is the reward value at the t-th step, T is the number of rounds of periodic execution, d0 and d1 respectively represent the minimum and maximum boundaries of the distance threshold, and d0 < d1 is satisfied; when d ≤ d1, the period ends;

[0071] Specifically,

[0072] When d ≤ d0, it is considered that the distance between the robotic arm and the target position is small enough. At this time, R t = 100, indicating that the task has been successfully completed;

[0073] [[ID=I9]]When d0 < d ≤ d1, the robotic arm is within the buffer range of the target position, and the reward value decreases linearly according to the change of ddd. Specifically to encourage the robotic arm to approach the target;

[0074] When d > d0, it is considered that the robotic arm is too far from the target, and the reward value is R t = -0.01 to reflect that the task is not ideal and continue training.

[0075] In addition, a curriculum learning strategy is introduced here:

[0076] In the initial stage, a larger target error range (d1 = 50mm, d0 = 20mm) is set to reduce the task complexity and accelerate the model's learning of the basic control strategy;

[0077] As the training progresses, gradually narrow the target range (d1 = 20mm, d0 = 10mm), increase the task difficulty and strengthen the model's fine control ability;

[0078] In the final stage, narrow the target range to industrial-level accuracy (d1 = 5mm, d0 = 1mm) to achieve high-precision inverse kinematics solution.

[0079] Figure 3-Figure 5 [[ID=I2]]Figure [X] shows the trend of reward change during the training process. It can be observed that when the parameters are d1 = 20mm and d0 = 10mm, the model shows the best training effect and the reward value reaches the highest. When d1 = 5mm and d0 = 1mm, there are obvious fluctuations or oscillations in the reward curve, indicating that the training process is more unstable at this time. Nevertheless, as the training progresses, the reward value finally tends to converge, indicating that the reinforcement learning algorithm can overcome the initial instability to a certain extent and finally achieve a stable optimization result.

[0080] S24. Model export;

[0081] Use the model saving and exporting functions of ML-Agents to export the trained reinforcement learning model to ONNX format.

[0082] S25. Model Integration and Verification

[0083] Integrate the exported ONNX model into Unity, load the model through the Barracuda plug-in and verify it.

[0084] Step 3: Use polynomial interpolation to plan the robot arm's motion trajectory and generate a smooth trajectory. The specific steps are as follows:

[0085] S31, randomly generate target points (x target ,y target ,z target ) Get the current joint angles (θ1, θ2, θ3, θ4, θ5, θ6) and end pose (x, y, z, roll, pitch, yaw) and pass them into the ONNX model obtained in step 2 to get the inverse angles of each joint (θ'1, θ'2, θ'3, θ'4, θ'5, θ'6)

[0086] S32. Use a fifth-order polynomial to generate the initial angle trajectory curve of each joint of the robotic arm to ensure the continuity and smoothness of the path.

[0087] S33. Import the interpolated trajectory into the digital twin platform and run the simulation on the digital robotic arm to verify the feasibility of the trajectory.

[0088] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for solving inverse kinematics and trajectory planning of a robotic arm based on digital twin technology, characterized in that: The following steps are involved: S1. Build a digital twin platform to achieve virtual-reality interaction; specifically, it includes setting up a virtual environment and establishing virtual-reality interactive communication; S2. Conduct reinforcement learning training based on the SAC algorithm to build an inverse kinematics solution model for the robotic arm. This includes state space design, action space design, gradual optimization of the inverse kinematics solution model through reward functions and curriculum learning strategies, model export, and model integration. S3, using polynomial interpolation to plan the robot arm's motion trajectory and generate a smooth trajectory; Step S2 is specifically as follows: S21. State Space Design: Build a reinforcement learning environment based on the Unity platform's ML-Agents toolkit and define the state space of the robotic arm. The state space includes: The current position of the end effector (x, y, z, roll, pitch, yaw) is used to characterize the control accuracy of the robotic arm; the current angle of each joint (θ1, ...θ n ), n depends on the degree of freedom of the manipulator; at the beginning of each cycle, a target position (x target ,y target ,z target ), as the core task goal of inverse kinematics solution; S22, Action Space Design: Using the ContinuousAction mechanism of ML-Agents, the angles of each joint of the robot arm (θ1, ...θ n ) as the output of the continuous action space, and set the action range to (-180°, 180°) to ensure that the action complies with the kinematic constraints of the joint; S23. Gradually optimize the inverse kinematics solution model through reward functions and curriculum learning strategies: During task execution, the Euclidean distance d between the end effector and the target point is calculated in real time, and a reward function is designed based on this distance. The specific reward setting is as follows: The total reward is as described in formula (2) Among them, R t is the reward value of step t, T is the number of steps in the cycle, d0 and d1 represent the minimum and maximum boundaries of the target error range, and d0 < d1; when d ≤ d0, the cycle ends; Specifically: When d≤d0, it is considered that the distance between the robot arm and the target position is small enough. t =100, indicating that the task has been successfully completed; When d0<d≤d1, the robot is within the buffer zone of the target position, and the reward value decreases linearly according to the change of d. Specifically, To encourage the robotic arm to move closer to the target; When d>d0, the robot is considered too far away from the target, and the reward value is R t =-0.01, reflecting that the task is not ideal and training continues; In addition, the course learning strategy is introduced: In the initial stage, a larger target error range is set to reduce task complexity and accelerate the model's learning of the basic control strategy; the larger target error range is d1 = 50 mm, d0 = 20 mm; As training progresses, the target error range is gradually narrowed, the task difficulty is increased, and the model's refined control capabilities are strengthened; the target error range is narrowed to d1 = 20 mm and d0 = 10 mm; In the final stage, the target error range is reduced to industrial-grade accuracy to achieve high-precision inverse kinematics solution; the industrial-grade accuracy is d1 = 5mm, d0 = 1mm; S24. Model export: Use the model save and export function of ML-Agents to export the trained reinforcement learning model to ONNX format; S25. Model integration: Integrate the exported ONNX format model into Unity and load the model through the Barracuda plug-in.

2. The method according to claim 1, wherein In step S1, the virtual environment is built by importing the URDF file of the physical robotic arm into Unity using the URDFImporter component to ensure that the structure, size, and joint layout of the digital robotic arm are exactly the same as those of the actual robotic arm; Then, the physical parameters of the digital robotic arm model, including the range of motion, inertia characteristics, and friction coefficient of the joints, were added to simulate the dynamic characteristics of the actual robotic arm. A corresponding virtual environment was created based on the experimental requirements, including target points, obstacles, and a 3D scene for the experiment. The specific steps to build virtual-reality interactive communication are: establishing a real-time communication interface between Unity and ROS through ROS-Connector and ROS-EndPoint to achieve state synchronization between the virtual environment and the actual robotic arm; Configure ROS topics for transmitting joint angles θ, end-point poses (x, y, z, roll, pitch, yaw), and sensor data, and define message formats to ensure efficient and accurate real-time communication. Finally, ensure that the operating status of the robotic arm can be transmitted to the virtual environment in real time, and at the same time return the feedback data of the virtual environment to the physical robotic arm.

3. The method according to claim 1, wherein Step S3 specifically includes: S31, randomly generate target points (x target ,y target ,z target ), get the current joint angles (θ1, ...θ n ) and the end pose (x, y, z, roll, pitch, yaw) are passed into the ONNX format model obtained in step 2 to obtain the inverse angle of each joint (θ1', ...θ n '); S32, using a fifth-order polynomial to generate the trajectory curve to ensure the continuity and smoothness of the path; S33. Import the interpolated trajectory into the digital twin platform and run the simulation on the digital robotic arm to verify the feasibility of the trajectory.

Images