A robot digital twin bidirectional motion synchronization method

Abstract

This invention relates to the field of industrial internet, and particularly to a method for bidirectional motion synchronization between a robot and its digital twin. The method includes the following steps: constructing a robot data model by analyzing the robot's characteristic information; importing the robot's 3D model into a digital twin system to form a virtual robot model; embedding the robot's kinematic model into the digital twin system as a controller for the virtual robot; constructing OPC UA bidirectional servers on both the physical and virtual robot sides; establishing data acquisition modules on both the physical and virtual robot sides to collect robot motion information and transmit it to the corresponding data nodes of the OPC UA servers; and employing a continuous motion synchronization method based on trapezoidal velocity control. This invention enables bidirectional motion synchronization control between a virtual robot and a physical robot based on the OPC UA protocol, achieving a new, intuitive, and accurate way of operating robots.

Description

Technical Field

[0001] This invention relates to the field of industrial internet, specifically to a method for synchronizing bidirectional motion of a robot digital twin. Background Technology

[0002] Robots are an important component of the industrial sector. In industrial manufacturing, robots have replaced most of the repetitive and complex industrial operations on production lines. This has not only significantly improved production efficiency and reduced production costs, but also reduced the probability of production line workers being exposed to high-risk environments. As the level of industrial automation and intelligence continues to improve, the application of robot technology has expanded from simple repetitive labor to intelligent, precise, and reconfigurable operations, which places higher demands on the intelligent control of robots.

[0003] Currently, robotics technology has brought about technological innovation in the manufacturing sector, but traditional robot control systems still have some limitations. In traditional robot control systems, the robot's motion state and motion information are often closed and not publicly accessible, making it difficult for operators to obtain real-time dynamic information about the robot. This results in an inability to intuitively evaluate the operational effects, and this lag increases operational risks and debugging difficulty. To overcome these limitations, digital twin technology has emerged. A digital twin is a virtual model that includes both a physical robot and a virtual robot. The virtual robot can map the state and behavior of the physical robot in real time. The digital twin system constructs a real-time, dynamic virtual robot by collecting data from the physical robot and its operating environment.

[0004] Currently, there are several main methods for building digital twin systems: One method is to implement the digital twin using specialized industrial software. This method is relatively simple, but it relies on the support of other specialized industrial software for robots and is also limited by the communication architecture specified by the industrial software, thus failing to achieve greater versatility. Another method is to build a digital twin based on the TCP / IP protocol stack. This method can enhance the versatility and cross-platform characteristics of the digital twin platform, but it lacks optimization and modeling for robots in industrial scenarios, enabling them to possess complete information acquisition, control methods, and alarm information.

[0005] Meanwhile, current digital twin systems for industrial robots lack methods for bidirectional motion synchronization control, making it difficult to achieve bidirectional motion mapping between physical and virtual robots in continuous motion scenarios. Summary of the Invention

[0006] To address the problems existing in the prior art, the purpose of this invention is to provide a method for bidirectional motion synchronization of robot digital twins. This method can provide a general robot digital twin implementation architecture, possessing universality, completeness, and cross-platform characteristics, and achieving a more efficient and intuitive robot control method.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A method for synchronizing bidirectional motion in a robot digital twin includes the following steps:

[0009] Step 1. Obtain multimodal information of the robot and construct a robot data model;

[0010] Step 2. Import the robot's 3D model into the digital twin system to create a virtual robot model;

[0011] Step 3. Establish a virtual robot controller, embed the robot kinematics model, and use the robot data model as the kinematics model data source to control the movement of the virtual robot model;

[0012] Step 4. Construct OPC UA bidirectional servers on both the physical robot side and the virtual robot side, instantiate the robot data model, and construct the data nodes of the bidirectional servers;

[0013] Step 5. Establish OPC UA data acquisition modules on both the physical robot side and the virtual robot side, and collect the robot's multimodal information in the instantiated form of the robot data model into the corresponding data nodes of the OPC UA bidirectional server;

[0014] Step 6. Construct a motion synchronization module based on a trapezoidal speed control continuous motion synchronization method to continuously synchronize motion data in the OPCUA bidirectional server to the physical or virtual robot, thereby realizing bidirectional motion synchronization between the digital twin physical robot and the virtual robot.

[0015] Furthermore, acquiring multimodal information about the robot includes: static robot data: robot factory data, robot calibration data; motion robot data: position, speed, and torque information of each axis of the robot collected with the help of encoders, grating rulers, and torque sensors; and peripheral device data: temperature, air pressure, current, and voltage signals of the motors during robot movement.

[0016] Furthermore, the robot data model adopts the idea of ​​object-oriented modeling to abstract the robot's multimodal information, summarize various types of multimodal data, and combine the abstraction of various control methods to obtain five major categories: static data, motion data, control methods, peripheral equipment data, and fault and notification time.

[0017] Furthermore, in step 2, a general 3D model of each joint of the robot is exported from the 3D computer-aided design software in the form of each joint being separated, and each joint model is imported into the digital twin system to form a virtual model of the robot.

[0018] Furthermore, in step 3, the virtual robot controller is connected to the robot joint models in the digital twin system, and can directly control the movement of each joint model in the digital twin system according to the various embedded robot kinematic models.

[0019] Furthermore, controlling the motion of the robot's virtual model includes virtual control targeting the input data to control the rotational joints to rotate around the joint axis and to control the linear joints to move along the linear motion direction.

[0020] Furthermore, the OPC UA bidirectional server includes a main thread that is an OPC UA server, exposing data and callable methods in the node to other clients. Simultaneously, the bidirectional server has a sub-thread that runs a sub-client, which connects to other bidirectional servers to control them, enabling simultaneous data output and input. Furthermore, the OPC UA data acquisition module includes an OPC UA client that collects multimodal information of physical and virtual robots, simultaneously synchronizing the robot motion multimodal data to the corresponding data node of the OPC UA bidirectional server specified by the module.

[0021] Furthermore, the trapezoidal velocity control includes a maximum acceleration of a. acc The maximum deceleration is a dec The maximum speed is v max The target position is x absolute The initial position is x now Calculate the uniform acceleration time t using the time-optimal strategy. acc Uniform deceleration time t dec Uniform motion time t vel The calculation process is as follows:

[0022]

[0023]

[0024]

[0025] If the velocity curve needs to form a trapezoidal shape, including three stages: acceleration, deceleration, and constant speed, then t vel If the number must be non-negative, then:

[0026]

[0027] When the above equation is satisfied, the displacement equation is:

[0028]

[0029] The velocity equation is:

[0030]

[0031] Furthermore, the continuous motion synchronization method based on trapezoidal velocity control includes an improvement on the traditional unit trapezoidal velocity control method, which becomes a continuous trapezoidal velocity control method. In the process of robot trapezoidal velocity control planning, a buffer is divided into the planning control process, so that it can receive further control information simultaneously during the motion. The new control information will be stored in the buffer. Under the premise of considering the current motion state, the motion method will replace the currently executed motion target with the waiting motion control target at a certain frequency.

[0032] Furthermore, the motion synchronization module includes an OPC UA client that takes the output of the aforementioned continuous motion control method based on trapezoidal velocity control as input, and calls the control method data node of the OPC UA bidirectional server with the target position, target velocity, and target acceleration of the target motion information as parameters at a specific control frequency, so that the above motion information can be transmitted and used to control the physical robot and virtual robot in the digital twin system.

[0033] In summary, the present invention has the following advantages:

[0034] This invention fully utilizes the position, speed, acceleration, and torque information of each axis of the robot, and collects multimodal information such as current, voltage, and temperature signals during motor movement, as well as visual information, to construct a robot data model. This data model is then instantiated as the OPC UA bidirectional server data node on both the physical robot side and the virtual robot side, constructing a complete digital twin system. This digital twin also has a continuous motion synchronization method based on trapezoidal speed control, realizing motion synchronization control between the physical robot and the virtual robot.

[0035] Compared with existing technologies, the beneficial effects of this invention are as follows: It proposes a complete and universal communication architecture for the virtual and physical sides of a digital twin based on OPC UA, realizes the complete restoration of the physical robot in a virtual environment by acquiring multimodal data of the physical robot, and constructs a bidirectional continuous motion control method for both the physical robot and the virtual robot based on this digital twin system framework. Attached Figure Description

[0036] Figure 1 This is a flowchart of the robot digital twin framework implementation method of the present invention.

[0037] Figure 2 This is a representation diagram of the robot data model of the present invention.

[0038] Figure 3 This is a schematic diagram of the digital twin bidirectional server communication architecture of the present invention.

[0039] Figure 4 This is a block diagram of the virtual robot motion mapping entity robot control according to the present invention.

[0040] Figure 5 This is a block diagram of the virtual robot motion mapping entity robot control according to the present invention. Detailed Implementation

[0041] The present invention will now be described in further detail.

[0042] Example 1:

[0043] like Figure 1 As shown, a method for synchronizing bidirectional motion in a robot digital twin includes the following steps:

[0044] In general, the robot digital twin system proposed in this invention consists of three parts: a virtual robot, a physical robot, and a communication architecture and control method between the virtual and physical robots. Firstly, regarding the construction of the virtual robot, the following steps are included:

[0045] (1) Determine the types of multimodal data required for building a robot data model in an industrial setting, and construct a robot data model for the OPC UA communication system. The representation diagram of this data model is as follows: Figure 2 As shown, the data model contains various data types, including the following five categories:

[0046] ① Robot static data: Robot factory data represented by equipment model and manufacturer markings, and robot calibration data represented by robot DH parameters;

[0047] ② Robot motion data: This data is the real-time parameter information of the robot when performing different tasks, including real-time motion data of the robot's single axis, including the position, speed, and torque of each axis, as well as global coordinate data, including the world coordinate position and world coordinate velocity of the end effector.

[0048] ③ Robot control methods: This data contains presets for various robot control methods, including single-axis control method information such as homing, Jog+ (positive slow motion), Jog- (negative slow motion), single-axis servo power-on, single-axis servo power-off, and single-axis emergency stop, as well as global control methods including global emergency stop, global homing, and encoder manual zeroing commands.

[0049] ④ Peripheral device data: This data comes from various sensors in the robot and its working environment, including temperature sensors, air pressure sensors, current and voltage sensors, as well as information about the robot's status and the surrounding environment.

[0050] ⑤ Faults and Notification Events: Records the robot's maintenance history and fault information and task status notifications during operation, including equipment fault events such as overload and collision faults, as well as notification events indicating completed control and currently executing process steps. This information is of great value for predicting future maintenance needs and timely identifying and resolving problems.

[0051] The aforementioned robot data model includes multimodal data that a digital twin system needs to accommodate. By standardizing and integrating similar types of data, a universal data model can be formed that can be used for robots of various types and from various manufacturers.

[0052] (2) From the 3D computer-aided design software, export the general 3D models of each joint of the robot in a joint-separated form, consistent with the actual physical characteristics of the physical robot. The export format is such as STEP or OBJ. These joints include, but are not limited to, joints of the arm, leg, wrist, and fingers. Import each joint model into the digital twin system to form a virtual robot model. Then, these general 3D models of the joints will be imported into the digital twin system. In the digital twin system, these individual joint models are recombined and configured according to the actual structure of the robot to form a complete, standardized, and consistent virtual robot model. This process involves not only adjusting the spatial position and orientation of the model, but also the motion constraints and motion angle range limits between joints. This virtual model is the virtual entity of the virtual robot in the digital twin system. It contains multimodal data, including static data, motion data, control methods, peripheral equipment data, fault and notification events, used to characterize and reproduce the physical robot.

[0053] (3) A virtual robot controller is established in the digital twin system, embedding a general robot kinematics model, and the robot data model is instantiated as a kinematics model data structure, thereby realizing the control of the robot virtual model to move in the digital twin system. The virtual control takes the input data as the target, controls the corresponding rotational joint to rotate around the joint axis by the angle specified by the input data, and controls the corresponding linear joint to move along the linear motion direction by the distance specified by the input data.

[0054] The construction of the communication architecture and control methods between virtual and real robots includes the following steps:

[0055] (1) Construct OPC UA bidirectional servers on both the physical robot side and the virtual robot side, instantiate the robot data model, and construct data nodes for the bidirectional servers. The communication architecture diagram of the bidirectional servers is shown below. Figure 3 As shown, this communication architecture comprises two parts: a physical side and a virtual side, both designed symmetrically, and includes the aforementioned virtual robot and virtual controller. The main thread of the OPC UA bidirectional server is an OPC UA server that exposes the node's data and callable methods to other clients. Simultaneously, this bidirectional server has a sub-thread that runs an OPC UA client. This client connects to other OPC UA bidirectional servers, enabling control over them and simultaneously handling both data input and output.

[0056] (2) An OPC UA data acquisition module is constructed on both the physical robot side and the virtual robot side. This module is an OPC UA client. The data acquisition module collects the multimodal information of the physical robot and the virtual robot, and synchronizes the robot motion multimodal data to the corresponding data node of the OPC UA bidirectional server specified by the module according to the pre-division of various data types. This module exists simultaneously on each physical robot side and the virtual robot side in the form of multiple instances, and collects the multimodal data of the physical robot and the virtual robot respectively and uploads it to the aforementioned OPC UA bidirectional server.

[0057] (3) There are two feasible control directions for the proposed digital twin data synchronization, as follows:

[0058] ① The motion of the physical robot drives the motion of the virtual robot, and the control block diagram is as follows: Figure 4 As shown. Where y referece This represents the target motion trajectory, which is input to the motion controller as the target motion trajectory. The motion controller outputs (Δu) R (Representing the change in the speed of the physical robot) The change in speed of the physical motor within a unit control cycle controls the movement of the physical motor, and the position (y) of the physical motor at this time.R The data will be transmitted to the virtual side via the OPC UA bidirectional server, serving as the target motion trajectory for controlling the virtual robot's movement. At this time, the virtual controller receives the aforementioned target position command and outputs the change in the virtual robot's motor speed Δu. V Control the virtual motor to move to position y V This data synchronization direction represents the process of the digital twin system reproducing the motion state of the physical robot. At this time, the physical robot is working in a normal, predetermined working mode, performing predetermined actions and operations, while the virtual robot obtains the motion state information of the physical robot through the aforementioned virtual-side OPC UA bidirectional server, and controls the movement of the virtual robot with the aforementioned virtual robot controller.

[0059] ② The virtual robot's motion drives the physical robot's motion; the control block diagram is as follows: Figure 5 As shown. Where y referece This represents the target motion trajectory, which serves as the input to the virtual controller. The virtual controller outputs the velocity change Δu of the virtual motor within a unit control cycle. V Controlling the movement of the virtual motor, the current position of the virtual motor (y) V The data will be transmitted to the physical side via the OPC UA bidirectional server, serving as the target motion trajectory to control the movement of the physical robot. At this time, the physical motion controller receives the aforementioned target position command and outputs the change in the physical robot's motor speed Δu. R Control the physical motor to move to position y R This data synchronization direction represents using motion data from the digital twin system as motion guidance, with the digital twin end controlling the motion of the physical robot. In this case, the motion data comes from the virtual robot, and the movement of the virtual robot in the digital twin system is controlled. Meanwhile, the physical robot obtains the motion information status of the virtual robot through the aforementioned physical-side OPC UA bidirectional server, and uses this data as guidance to control the movement of the physical robot.

[0060] (4) Construct a motion synchronization module based on a continuous motion synchronization method using trapezoidal velocity control to continuously synchronize motion data from the OPC UA bidirectional server to the physical or virtual robot, thereby achieving bidirectional motion synchronization between the digital twin physical robot and the virtual robot. The continuous motion synchronization method based on trapezoidal velocity control improves upon the traditional unit trapezoidal velocity control method, becoming a continuous trapezoidal velocity control method. During the robot's trapezoidal velocity control planning process, a buffer is partitioned within the planning control process, allowing the robot to simultaneously receive further control information during its controlled motion. The new control information is stored in the buffer. This motion method, considering the current motion state, updates the currently executing motion target with the waiting motion control target at a certain frequency. Based on trapezoidal velocity planning, this strategy re-plans the motion strategy in real time each time a new target point is received, according to the current robot motion, to adapt to the new target point. Assume the robot's current position is x when the new point information arrives. now2 The current speed is v now2 The new target location is x new The new maximum speed of motion is v max2 The maximum acceleration is a acc2 The maximum deceleration is a dec2 The above problem can then be transformed into:

[0061] With x now2 v is the current initial position. now Given an initial velocity, plan to run to x. new The trapezoidal velocity programming problem is solved, and the planning process is as follows:

[0062] It can calculate the uniform acceleration time t under the current target using the time-optimal strategy. acc2 Uniform deceleration time t dec2 Uniform motion time t vel2 as follows:

[0063]

[0064]

[0065]

[0066] If the quadratic programming process satisfies t vel2 If ≥0, then the velocity trajectory formed after the above quadratic programming is still a trapezoid, containing three stages: acceleration, deceleration, and constant speed, that is:

[0067]

[0068] When the above equation is satisfied, the displacement equation can be calculated as follows:

[0069]

[0070] The velocity equation is:

[0071]

[0072] The planning method described above is based on trapezoidal dynamic programming. It can directly input the next desired motion point before the robot's previous motion has stopped, and re-plan the motion using the current position and speed as initial conditions. This satisfies the requirement of continuously inputting new target points at a certain sampling frequency under the synchronous motion control framework mentioned above, thereby achieving continuous bidirectional motion synchronization.

[0073] The present invention has the following beneficial effects:

[0074] This invention proposes a method for establishing robot data models for digital twins, which can describe the multimodal data of robots in digital twin systems in a general and efficient manner.

[0075] The OPC UA bidirectional communication system proposed in this invention can realize cross-device and cross-platform information collection and bidirectional data transmission. At the same time, an OPC UA bidirectional server is established, enabling the module to collect data from the same side and simultaneously acquire and synchronize data from the opposite side.

[0076] The continuous motion synchronization method based on trapezoidal speed control proposed in this invention can achieve efficient continuous motion synchronization in a digital twin bidirectional motion synchronization environment, enabling the motion of the physical robot to be synchronized to the virtual robot, and the motion of the virtual robot to be synchronized to the physical robot, thus realizing a new way of intuitive and accurate robot operation.

[0077] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for synchronizing bidirectional motion of a robot digital twin, characterized in that: Includes the following steps, Step 1. Obtain multimodal information of the robot and construct a robot data model; Step 2. Import the robot's 3D model into the digital twin system to create a virtual robot model; Step 3. Establish a virtual robot controller, embed the robot kinematics model, and use the robot data model as the kinematics model data source to control the movement of the virtual robot model; Step 4. Construct OPC UA bidirectional servers on both the physical robot side and the virtual robot side, instantiate the robot data model, and construct the data nodes of the OPC UA bidirectional server; Step 5. Establish OPC UA data acquisition modules on both the physical robot side and the virtual robot side, and collect the robot's multimodal information in the instantiated form of the robot data model into the corresponding data nodes of the OPC UA bidirectional server; Step 6. Construct a motion synchronization module based on the continuous motion synchronization method of trapezoidal speed control, and continuously synchronize the motion data in the OPC UA bidirectional server to the physical or virtual robot, thereby realizing bidirectional motion synchronization between the digital twin physical robot and the virtual robot; Acquiring multimodal information about the robot includes static data, motion data, and peripheral device data. Robot static data includes: robot manufacturing data and robot calibration data; Robot motion data includes: position, speed, and torque information of each axis of the robot collected using encoders, linear encoders, and torque sensors; Peripheral device data includes: temperature and air pressure during robot movement, and current and voltage signals during motor movement; The robot data model adopts the idea of ​​object-oriented modeling, abstracts the multimodal information of the robot, summarizes various types of multimodal data, and combines the abstraction of various control methods to obtain five categories: static data, motion data, control methods, peripheral equipment data, and fault and notification time. The continuous motion synchronization method based on trapezoidal velocity control improves upon the traditional unit trapezoidal velocity control method, becoming a continuous trapezoidal velocity control method. During the robot's trapezoidal velocity control planning process, a buffer is defined, allowing the robot to simultaneously receive further control information during motion. This new control information is stored in the buffer. Considering the current motion state, this continuous motion synchronization method replaces the currently executing motion target with the waiting motion target at a certain frequency.

2. The method for synchronizing bidirectional motion of a robot digital twin according to claim 1, characterized in that: In step 2, a general 3D model of each joint of the robot is exported from the 3D computer-aided design software in the form of each joint being separated, and the general 3D model of each joint is imported into the digital twin system to form a virtual model of the robot.

3. The method for synchronizing bidirectional motion of a robot digital twin according to claim 2, characterized in that: In step 3, the virtual robot controller is connected to the universal 3D model of each joint of the robot in the digital twin system, and can directly control the universal 3D model of each joint of the robot to move in the digital twin system according to the various robot kinematic models embedded.

4. The method for synchronizing bidirectional motion of a robot digital twin according to claim 1, characterized in that: The main thread of the OPC UA bidirectional server is an OPC UA server, which has a sub-thread that runs a client.

5. The method for synchronizing bidirectional motion of a robot digital twin according to claim 1, characterized in that: The OPC UA data acquisition module is an OPC UA client that collects multimodal information of physical and virtual robots and synchronizes the robot motion multimodal information to the corresponding data node of the OPC UA bidirectional server specified by the OPC UA data acquisition module.

6. The method for synchronizing bidirectional motion of a robot digital twin according to claim 1, characterized in that: Trapezoidal velocity control includes a maximum acceleration of The maximum deceleration is The maximum speed is The target location is The initial position is Calculate uniform acceleration time using the time-optimal strategy Uniform deceleration time Uniform motion time The calculation process is as follows: ； If the velocity curve needs to form a trapezoidal shape, including three stages: acceleration, deceleration, and constant speed, then If the number must be non-negative, then: ； When the above equation is satisfied, the displacement equation is: The velocity equation is: 。 7. The method for synchronizing bidirectional motion of a robot digital twin according to claim 1, characterized in that: The motion synchronization module is an OPC UA client. By taking the output of the aforementioned continuous motion synchronization method based on trapezoidal velocity control as input, it calls the control method data node of the OPC UA bidirectional server with the target position, target velocity, and target acceleration of the target motion information as parameters at a specific control frequency. This enables the aforementioned target motion information to be transmitted and used to control the physical robot and virtual robot in the digital twin system.

Images