Digital twinning digital entity modeling method for industrial robot

Through the combination of deep learning models and 3D geometric models, the digital solid model of the robot arm is constructed and optimized, which solves the time and instability problems of traditional control systems, realizes real-time monitoring and optimization of the robot arm, and improves intelligent manufacturing Application efficiency and safety.

CN119077736AInactive Publication Date: 2024-12-06CHINA UNIV OF MINING & TECH
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411350590.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-26
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional robotic arm control systems rely on physical experiments and simulation tests, which have problems such as time-consuming, high cost and unstable results. It is difficult to visually present dynamic behavior under complex working conditions, and it is difficult to achieve real-time monitoring and optimization.

Method used

The deep learning model is used to combine 3D geometric model to build a digital entity model on the Unity platform, update it in real time and adjust it dynamically, and build and optimize the virtual model of the robot arm through multi-branch neural networks and RRT algorithms to achieve synchronous operation with physical entities.

Benefits of technology

Improve the accuracy and reliability of the model, realize real-time data updates and dynamic adjustments, improve the intelligence level of the robotic arm, simplify the design and debugging process, reduce risks and energy consumption in production, and improve operational efficiency and safety. sex.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119077736A_ABST
    Figure CN119077736A_ABST
Patent Text Reader

Abstract

The invention discloses a digital twinning digital entity modeling method for an industrial robot, which comprises the following steps: acquiring real-time operation data of a physical entity of a mechanical arm and an expected joint angle, and preprocessing the data to ensure the validity and consistency of the input data; based on the preprocessed data, a deep learning model of the mechanical arm is constructed; the deep learning model is combined with the 3D geometric model, and a digital entity model of the mechanical arm is constructed on a Unity platform; in the operation process of the mechanical arm, data of the digital entity model are updated in real time, and the model is dynamically adjusted and self-perfected; the digital twin modeling method can realize simulation and real-time optimization of the industrial robot, is suitable for the fields of industrial automation, intelligent manufacturing and the like, has a wide application prospect, can improve the operation efficiency and the safety, and optimizes the cooperation between the robots at the same time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of intelligent manufacturing and robotics technology, and in particular to a method for digital twin modeling of an industrial robot. Background Art

[0002] Digital twin technology creates virtual models of physical devices to achieve real-time monitoring, simulation, prediction and optimization of physical entities. In the field of robotic arms, the application of digital twins provides a new technical path for precise control and efficient operation.

[0003] Traditional robotic arm control systems usually rely on physical experiments and simulation tests for verification. However, these methods have obvious limitations. Physical experiments are not only time-consuming and costly, but also easily interfered by environmental factors, resulting in unstable repeatability and accuracy of experimental results. Although numerical simulation can simulate the movement of the robotic arm, it is difficult to intuitively present the dynamic behavior of the robotic arm under complex working conditions. The digital entity modeling technology of the digital twin robotic arm, by establishing a virtual model of the robotic arm, realizes the synchronous operation of virtual and reality. This technology can not only reflect the physical state of the robotic arm in real time, but also simulate its motion trajectory and mechanical properties, so as to perform predictive maintenance, optimize the operation path and improve the reliability of the control algorithm. This modeling technology helps to improve the intelligence level of the robotic arm, simplify the design and debugging process, while reducing potential risks and energy consumption in production, further promoting the application and popularization of robotic arms in intelligent manufacturing. Summary of the invention

[0004] In view of the above-mentioned technical deficiencies, the purpose of the present invention is to provide a digital twin digital entity modeling method for industrial robots, which can solve the problem of constructing digital entity models during the operation of industrial robots, can realize the simulation and real-time optimization of industrial robots, is suitable for industrial automation, intelligent manufacturing and other fields, has broad application prospects, can improve work efficiency and safety, and optimize the collaboration and cooperation between robots.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] The present invention provides a digital twin digital entity modeling method for an industrial robot, comprising the following steps:

[0007] S1. Obtain the real-time operation data of the physical entity of the robot arm (including the real-time operation speed v and angle θ of the joint) and the expected joint angle θ c , and pre-process these data to ensure the validity and consistency of the input data;

[0008] S2. Build a deep learning model of the robotic arm based on the preprocessed data;

[0009] S3, combines the deep learning model with the 3D geometric model to build a digital solid model of the robotic arm on the Unity platform;

[0010] S4, during the operation of the robot arm, updates the data of the digital entity model in real time, and dynamically adjusts and self-improves the model.

[0011] Preferably, step S1 specifically includes:

[0012] S11, real-time collection of the joint angle θ and speed v operation data of the robot arm;

[0013] S12, monitoring the environment information around the robot arm, including but not limited to obstacle position P and temperature T parameters;

[0014] S13, calculate and record the desired joint angle θ of the robot arm through the RRT algorithm c ;

[0015] S14, collect the actual joint angle θ, speed v, environmental information (P, T) and the expected joint angle θ generated by trajectory planning c The data is transmitted to the controller for preliminary data processing and filtering;

[0016] S15. The controller stores the processed data in a database for use in subsequent steps of deep learning model training and digital entity model construction.

[0017] Preferably, step S2 specifically includes:

[0018] S21, the expected joint angle θ preprocessed in step 1 c , joint velocity v and environmental information (P, T) are input into different input layers of the multi-branch neural network respectively;

[0019] S22. In a multi-branch neural network, an independent sub-network is set for each input to perform feature extraction and analysis;

[0020] S23, merging the features extracted by each sub-network at the fusion layer of the network to form an overall digital model of the robotic arm;

[0021] S24. During the model training process, the mean square error (MSE) is used as the loss function. The deviation between the joint angle θ′ predicted by the neural network and the actual observed value θ is calculated, and then the network parameters are adjusted to ensure the accuracy of the model.

[0022] S25. When the model reaches the preset loss value on the validation set, save the model in onnx format for use in Unity.

[0023] Preferably, step S3 specifically includes:

[0024] S31. Use the Barracuda plug-in of Unity to import the onnx model trained in step S2 into the Unity environment;

[0025] S32. Import the URDF model of the robotic arm through URDF Importer and load it into the Unity virtual engine to form a virtual entity robotic arm;

[0026] S33. In the Unity scene, the actual joint angles, joint velocities, and other input factors required by the model in the Unreal Engine are passed to the Barracuda model;

[0027] S34, Barracuda model calculates based on input data and outputs predicted actual joint angles;

[0028] S35. Assign the actual joint angle output by the Barracuda model to the virtual entity in Unity so that it simulates and tracks the motion behavior state of the real robotic arm.

[0029] Preferably, step S5 includes the following contents:

[0030] S41, using the same method as step S1 to collect the joint angle θ, speed v, and expected joint angle θ′ operation data of the robot arm; and environmental information including the obstacle position P and temperature T around the robot arm;

[0031] S42. Implement model retraining and fine-tuning, and use new data to incrementally train the model to adapt to changes in the robot's operating environment and performance degradation, ensuring that the digital twin model maintains a high degree of consistency with the physical entity;

[0032] S43. Through the iterative update mechanism, combined with greedy local search, the adjustable parameters of the model are continuously optimized to achieve consistency measurement between the model and the physical entity, and improve the real-time self-correction capability of the digital twin model;

[0033] S44. Finally, the updated digital entity model data is stored in the database for subsequent model training, analysis and decision support, forming a closed-loop system of continuous optimization and self-improvement.

[0034] The beneficial effects of the present invention are:

[0035] (1) This method collects robot data and constructs a model on the Unity platform, combines the 3D geometric model to form a digital solid model, integrates the dynamic behavior of the robot and the impact of the environment, and improves the accuracy and reliability of the model.

[0036] (2) This method realizes real-time data updating and dynamic adjustment: by collecting and updating data in real time, the digital entity model can be dynamically adjusted according to the operating status of the physical entity, ensuring its synchronization and consistency with the physical entity.

[0037] (3) Providing an extensible application framework: The modeling method and implementation scheme proposed in this application have good scalability and versatility, and can be extended and applied to the construction of digital twin models of other types of industrial equipment, providing a strong reference for research and application in related fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A schematic diagram of a mechanical arm model provided by an embodiment of the present invention;

[0040] Figure 2 Schematic diagram of the network structure for modeling the robotic arm. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] This embodiment provides a digital twin digital entity modeling method for an industrial robot, comprising the following steps:

[0043] Step 1: Use the 6-DOF robotic arm of Zhongke Shengu ROCR6 (see Figure 1 ), obtain the real-time operation data and expected joint angles of the physical entity of the robot arm, and perform data preprocessing; specifically:

[0044] S11, collect the operation data of each joint angle θ, speed v, etc. of the robot arm in real time through the 17-bit absolute encoder

[0045] S12. Monitor the environment information around the robot arm through temperature and humidity sensors and depth cameras, including but not limited to obstacle position P and temperature T parameters;

[0046] S13, use the RRT algorithm to plan the trajectory of any point and record the expected joint angles θ during the operation of the robot c ;

[0047] S14, collect the actual joint angle θ, joint velocity v, environmental information (P, T) and the expected joint angle θ generated by the computer c The data is transmitted to the controller or similar central control system through the data acquisition system for preliminary data processing and filtering to ensure the accuracy and validity of the data;

[0048] S15, storing the processed data in a database for use in subsequent steps of deep learning model training and digital entity model construction;

[0049] Step 2: Build a deep learning model of the robotic arm based on the processed data; including the following:

[0050] S21, the desired joint angle θ processed in step 1 c , joint velocity v and environmental information (P, T) are input into different input layers of the multi-branch neural network respectively;

[0051] S22. In a multi-branch neural network, an independent sub-network is set for each input to perform feature extraction and analysis;

[0052] S23, the features extracted by each sub-network are merged in the fusion layer of the network to form the overall digital model of the robotic arm. The model structure is as follows: Figure 2 As shown;

[0053] S24. During the model training process, the mean square error (MSE) is used as the loss function. The deviation between the joint angle θ′ predicted by the neural network and the actual observed value θ is calculated, and then the network parameters are adjusted to ensure the accuracy of the model.

[0054] S25. When the model reaches the preset loss value on the validation set, save the model in onnx format for use in Unity.

[0055] Step 3: Combine the deep learning model with the 3D geometric model to build a digital solid model of the robotic arm on the Unity platform, which includes the following:

[0056] S31. Use the Barracuda plug-in of Unity to import the onnx model trained in step 2 into the Unity environment.

[0057] S32. Import the URDF model of the robotic arm through URDF Importer and load it into the Unity virtual engine to form a virtual entity robotic arm;

[0058] S33. In the Unity scene, the actual joint angles, joint velocities, and other input factors (such as environmental information) required by the model in the virtual engine are passed to the Barracuda model;

[0059] S34, the Barracuda model calculates according to the input data and outputs the predicted actual joint angle; S35, the actual joint angle output by the Barracuda model is assigned to the virtual entity in Unity, so that it simulates the motion state of the real robotic arm;

[0060] Step 4: During the operation of the robot arm, the data of the digital entity model is updated in real time, and the model is dynamically adjusted and self-improved, including the following:

[0061] S41, using the same method as step 1, real-time data collection of the robot arm. This includes real-time collection of operating data such as the joint angle θ, speed v, and expected joint angle θ′ of the robot arm. At the same time, the environmental sensors around the robot arm continuously monitor environmental information including obstacle position P, temperature T, etc.

[0062] S42. Implement model retraining and fine-tuning, and use new data to incrementally train the model to adapt to changes in the robot's operating environment and performance degradation, ensuring that the digital twin model maintains a high degree of consistency with the physical entity.

[0063] S43. Through the iterative update mechanism combined with greedy local search, the adjustable parameters of the model are continuously optimized to achieve consistency measurement between the model and the physical entity, thereby improving the real-time self-correction capability of the digital twin model.

[0064] S44. Finally, the updated digital entity model data is stored in the database for subsequent model training, analysis and decision support, forming a closed-loop system of continuous optimization and self-improvement.

[0065] This method can solve the problem of constructing digital entity models during the operation of industrial robots, can realize the simulation and real-time optimization of industrial robots, is suitable for industrial automation, intelligent manufacturing and other fields, has broad application prospects, can improve operating efficiency and safety, and optimize the collaboration between robots.

[0066] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A digital twin digital entity modeling method for an industrial robot, characterized in that: The following steps are involved: S1. Obtain the real-time operation data of the physical entity of the robot and the expected joint angle θ c ,The real-time operation data includes the real-time operation speed v and angle θ of the joint, and these data are pre-processed to ensure the validity and consistency of the input data; S2. Build a deep learning model of the robotic arm based on the preprocessed data; S3, combines the deep learning model with the 3D geometric model to build a digital solid model of the robotic arm on the Unity platform; S4, during the operation of the robot arm, updates the data of the digital entity model in real time, and dynamically adjusts and self-improves the model.

2. A digital twin digital entity modeling method for an industrial robot according to claim 1, characterized in that: Step S1 specifically includes: S11, real-time collection of the joint angle θ and speed v operation data of the robot arm; S12, monitoring the environment information around the robot arm, including but not limited to obstacle position P and temperature T parameters; S13, calculate and record the desired joint angle θ of the robot arm through the RRT algorithm c ; S14, collect the actual joint angle θ, speed v, environmental information (P, T) and the expected joint angle θ generated by trajectory planning c The data is transmitted to the controller for preliminary data processing and filtering; S15. The controller stores the processed data in a database for use in subsequent steps of deep learning model training and digital entity model construction.

3. A digital twin digital entity modeling method for an industrial robot according to claim 1, characterized in that: Step S2 specifically includes: S21, the expected joint angle θ preprocessed in step 1 c , joint velocity v and environmental information (P, T) are input into different input layers of the multi-branch neural network respectively; S22. In a multi-branch neural network, an independent sub-network is set for each input to perform feature extraction and analysis; S23, merging the features extracted by each sub-network at the fusion layer of the network to form an overall digital model of the robotic arm; S24. During the model training process, the mean square error (MSE) is used as the loss function. The deviation between the joint angle θ′ predicted by the neural network and the actual observed value θ is calculated, and then the network parameters are adjusted to ensure the accuracy of the model. S25. When the model reaches the preset loss value on the validation set, save the model in onnx format for use in Unity.

4. A digital twin digital entity modeling method for an industrial robot according to claim 3, characterized in that: Step S3 specifically includes: S31. Use the Barracuda plug-in of Unity to import the onnx model trained in step S2 into the Unity environment; S32. Import the URDF model of the robotic arm through URDF Importer and load it into the Unity virtual engine to form a virtual entity robotic arm; S33. In the Unity scene, the actual joint angles, joint velocities, and other input factors required by the model in the Unreal Engine are passed to the Barracuda model; S34, Barracuda model calculates based on input data and outputs predicted actual joint angles; S35. Assign the actual joint angle output by the Barracuda model to the virtual entity in Unity so that it simulates and tracks the motion behavior state of the real robotic arm.

5. A digital twin digital entity modeling method for an industrial robot according to claim 4, characterized in that: Step S4 includes the following contents: S41, using the same method as step S1 to collect the joint angle θ, speed v, and expected joint angle θ′ operation data of the robot arm; and environmental information including the obstacle position P and temperature T around the robot arm; S42. Implement model retraining and fine-tuning, and use new data to incrementally train the model to adapt to changes in the robot's operating environment and performance degradation, ensuring that the digital twin model maintains a high degree of consistency with the physical entity; S43. Through the iterative update mechanism, combined with greedy local search, the adjustable parameters of the model are continuously optimized to achieve consistency measurement between the model and the physical entity, and improve the real-time self-correction capability of the digital twin model; S44. Finally, the updated digital entity model data is stored in the database for subsequent model training, analysis and decision support, forming a closed-loop system of continuous optimization and self-improvement.

Citation Information

Patent Citations

  • Operation optimization method for long-distance belt conveyor driven by digital twinning

    CN114117856A

  • Industrial mechanical arm control method and system based on digital twin technology

    CN115446867A

  • High-precision mechanical arm control method and system based on reinforcement learning

    CN115781685A

  • Digital twin mechanical arm reinforcement learning training method and system based on Unity

    CN117798928A

  • Industrial robot digital twin model pose precision compensation method, device and system

    CN118682764A