Digital twinning digital entity modeling method for industrial robot
Through the combination of deep learning models and 3D geometric models, the digital solid model of the robot arm is constructed and optimized, which solves the time and instability problems of traditional control systems, realizes real-time monitoring and optimization of the robot arm, and improves intelligent manufacturing Application efficiency and safety.
Patent Information
- Application Number
- CN202411350590.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-26
- Publication Date
- 2024-12-06
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional robotic arm control systems rely on physical experiments and simulation tests, which have problems such as time-consuming, high cost and unstable results. It is difficult to visually present dynamic behavior under complex working conditions, and it is difficult to achieve real-time monitoring and optimization.
The deep learning model is used to combine 3D geometric model to build a digital entity model on the Unity platform, update it in real time and adjust it dynamically, and build and optimize the virtual model of the robot arm through multi-branch neural networks and RRT algorithms to achieve synchronous operation with physical entities.
Improve the accuracy and reliability of the model, realize real-time data updates and dynamic adjustments, improve the intelligence level of the robotic arm, simplify the design and debugging process, reduce risks and energy consumption in production, and improve operational efficiency and safety. sex.
Smart Images

Figure CN119077736A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent manufacturing and robotics technology, and in particular to a method for digital twin modeling of an industrial robot. Background Art
[0002] Digital twin technology creates virtual models of physical devices to achieve real-time monitoring, simulation, prediction and optimization of physical entities. In the field of robotic arms, the application of digital twins provides a new technical path for precise control and efficient operation.
[0003] Traditional robotic arm control systems usually rely on physical experiments and simulation tests for verification. However, these methods have obvious limitations. Physical experiments are not only time-consuming and costly, but also easily interfered by environmental factors, resulting in unstable repeatability and accuracy of experimental results. Although numerical simulation can simulate the movement of the robotic arm, it is difficult to intuitively present the dynamic behavior of the robotic arm under complex working conditions. The digital entity modeling technology of the digital twin robotic arm, by establishing a virtual model of the robotic arm, realizes the synchronous operation of virtual and reality. This technology can not only reflect the physical state of the robotic arm in real time, but also simulate its motion trajectory and mechanical properties, so as to perform predictive maintenance, optimize the operation path and improve the reliability of the control algorithm. This modeling technology helps to improve the intelligence level of the robotic arm, simplify the design and debugging process, while reducing potential risks and energy consumption in production, further promoting the application and popularization of robotic arms in intelligent manufacturing. Summary of the invention
[0004] In view of the above-mentioned technical deficiencies, the purpose of the present invention is to provide a digital twin digital entity modeling method for industrial robots, which can solve the problem of constructing digital entity models during the operation of industrial robots, can realize the simulation and real-time optimization of industrial robots, is suitable for industrial automation, intelligent manufacturing and other fields, has broad application prospects, can improve work efficiency and safety, and optimize the collaboration and cooperation between robots.
[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:
[0006] The present invention provides a digital twin digital entity modeling method for an industrial robot, comprising the following steps:
[0007] S1. Obtain the real-time operation data of the physical entity of the robot arm (including the real-time operation speed v and angle θ of the joint) and the expected joint angle θ c , and pre-process these data to ensure the validity and consistency of the input data;
[0008] S2. Build a deep learning model of the robotic arm based on the preprocessed data;
[0009] S3, combines the deep learning model with the 3D geometric model to build a digital solid model of the robotic arm on the Unity platform;
[0010] S4, during the operation of the robot arm, updates the data of the digital entity model in real time, and dynamically adjusts and self-improves the model.
[0011] Preferably, step S1 specifically includes:
[0012] S11, real-time collection of the joint angle θ and speed v operation data of the robot arm;
[0013] S12, monitoring the environment information around the robot arm, including but not limited to obstacle position P and temperature T parameters;
[0014] S13, calculate and record the desired joint angle θ of the robot arm through the RRT algorithm c ;
[0015] S14, collect the actual joint angle θ, speed v, environmental information (P, T) and the expected joint angle θ generated by trajectory planning c The data is transmitted to the controller for preliminary data processing and filtering;
[0016] S15. The controller stores the processed data in a database for use in subsequent steps of deep learning model training and digital entity model construction.
[0017] Preferably, step S2 specifically includes:
[0018] S21, the expected joint angle θ preprocessed in step 1 c , joint velocity v and environmental information (P, T) are input into different input layers of the multi-branch neural network respectively;
[0019] S22. In a multi-branch neural network, an independent sub-network is set for each input to perform feature extraction and analysis;
[0020] S23, merging the features extracted by each sub-network at the fusion layer of the network to form an overall digital model of the robotic arm;
[0021] S24. During the model training process, the mean square error (MSE) is used as the loss function. The deviation between the joint angle θ′ predicted by the neural network and the actual observed value θ is calculated, and then the network parameters are adjusted to ensure the accuracy of the model.
[0022] S25. When the model reaches the preset loss value on the validation set, save the model in onnx format for use in Unity.
[0023] Preferably, step S3 specifically includes:
[0024] S31. Use the Barracuda plug-in of Unity to import the onnx model trained in step S2 into the Unity environment;
[0025] S32. Import the URDF model of the robotic arm through URDF Importer and load it into the Unity virtual engine to form a virtual entity robotic arm;
[0026] S33. In the Unity scene, the actual joint angles, joint velocities, and other input factors required by the model in the Unreal Engine are passed to the Barracuda model;
[0027] S34, Barracuda model calculates based on input data and outputs predicted actual joint angles;
[0028] S35. Assign the actual joint angle output by the Barracuda model to the virtual entity in Unity so that it simulates and tracks the motion behavior state of the real robotic arm.
[0029] Preferably, step S5 includes the following contents:
[0030] S41, using the same method as step S1 to collect the joint angle θ, speed v, and expected joint angle θ′ operation data of the robot arm; and environmental information including the obstacle position P and temperature T around the robot arm;
[0031] S42. Implement model retraining and fine-tuning, and use new data to incrementally train the model to adapt to changes in the robot's operating environment and performance degradation, ensuring that the digital twin model maintains a high degree of consistency with the physical entity;
[0032] S43. Through the iterative update mechanism, combined with greedy local search, the adjustable parameters of the model are continuously optimized to achieve consistency measurement between the model and the physical entity, and improve the real-time self-correction capability of the digital twin model;
[0033] S44. Finally, the updated digital entity model data is stored in the database for subsequent model training, analysis and decision support, forming a closed-loop system of continuous optimization and self-improvement.
[0034] The beneficial effects of the present invention are:
[0035] (1) This method collects robot data and constructs a model on the Unity platform, combines the 3D geometric model to form a digital solid model, integrates the dynamic behavior of the robot and the impact of the environment, and improves the accuracy and reliability of the model.
[0036] (2) This method realizes real-time data updating and dynamic adjustment: by collecting and updating data in real time, the digital entity model can be dynamically adjusted according to the operating status of the physical entity, ensuring its synchronization and consistency with the physical entity.
[0037] (3) Providing an extensible application framework: The modeling method and implementation scheme proposed in this application have good scalability and versatility, and can be extended and applied to the construction of digital twin models of other types of industrial equipment, providing a strong reference for research and application in related fields. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0039] Figure 1 A schematic diagram of a mechanical arm model provided by an embodiment of the present invention;
[0040] Figure 2 Schematic diagram of the network structure for modeling the robotic arm. DETAILED DESCRIPTION
[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0042] This embodiment provides a digital twin digital entity modeling method for an industrial robot, comprising the following steps:
[0043] Step 1: Use the 6-DOF robotic arm of Zhongke Shengu ROCR6 (see Figure 1 ), obtain the real-time operation data and expected joint angles of the physical entity of the robot arm, and perform data preprocessing; specifically:
[0044] S11, collect the operation data of each joint angle θ, speed v, etc. of the robot arm in real time through the 17-bit absolute encoder
[0045] S12. Monitor the environment information around the robot arm through temperature and humidity sensors and depth cameras, including but not limited to obstacle position P and temperature T parameters;
[0046] S13, use the RRT algorithm to plan the trajectory of any point and record the expected joint angles θ during the operation of the robot c ;
[0047] S14, collect the actual joint angle θ, joint velocity v, environmental information (P, T) and the expected joint angle θ generated by the computer c The data is transmitted to the controller or similar central control system through the data acquisition system for preliminary data processing and filtering to ensure the accuracy and validity of the data;
[0048] S15, storing the processed data in a database for use in subsequent steps of deep learning model training and digital entity model construction;
[0049] Step 2: Build a deep learning model of the robotic arm based on the processed data; including the following:
[0050] S21, the desired joint angle θ processed in step 1 c , joint velocity v and environmental information (P, T) are input into different input layers of the multi-branch neural network respectively;
[0051] S22. In a multi-branch neural network, an independent sub-network is set for each input to perform feature extraction and analysis;
[0052] S23, the features extracted by each sub-network are merged in the fusion layer of the network to form the overall digital model of the robotic arm. The model structure is as follows: Figure 2 As shown;
[0053] S24. During the model training process, the mean square error (MSE) is used as the loss function. The deviation between the joint angle θ′ predicted by the neural network and the actual observed value θ is calculated, and then the network parameters are adjusted to ensure the accuracy of the model.
[0054] S25. When the model reaches the preset loss value on the validation set, save the model in onnx format for use in Unity.
[0055] Step 3: Combine the deep learning model with the 3D geometric model to build a digital solid model of the robotic arm on the Unity platform, which includes the following:
[0056] S31. Use the Barracuda plug-in of Unity to import the onnx model trained in step 2 into the Unity environment.
[0057] S32. Import the URDF model of the robotic arm through URDF Importer and load it into the Unity virtual engine to form a virtual entity robotic arm;
[0058] S33. In the Unity scene, the actual joint angles, joint velocities, and other input factors (such as environmental information) required by the model in the virtual engine are passed to the Barracuda model;
[0059] S34, the Barracuda model calculates according to the input data and outputs the predicted actual joint angle; S35, the actual joint angle output by the Barracuda model is assigned to the virtual entity in Unity, so that it simulates the motion state of the real robotic arm;
[0060] Step 4: During the operation of the robot arm, the data of the digital entity model is updated in real time, and the model is dynamically adjusted and self-improved, including the following:
[0061] S41, using the same method as step 1, real-time data collection of the robot arm. This includes real-time collection of operating data such as the joint angle θ, speed v, and expected joint angle θ′ of the robot arm. At the same time, the environmental sensors around the robot arm continuously monitor environmental information including obstacle position P, temperature T, etc.
[0062] S42. Implement model retraining and fine-tuning, and use new data to incrementally train the model to adapt to changes in the robot's operating environment and performance degradation, ensuring that the digital twin model maintains a high degree of consistency with the physical entity.
[0063] S43. Through the iterative update mechanism combined with greedy local search, the adjustable parameters of the model are continuously optimized to achieve consistency measurement between the model and the physical entity, thereby improving the real-time self-correction capability of the digital twin model.
[0064] S44. Finally, the updated digital entity model data is stored in the database for subsequent model training, analysis and decision support, forming a closed-loop system of continuous optimization and self-improvement.
[0065] This method can solve the problem of constructing digital entity models during the operation of industrial robots, can realize the simulation and real-time optimization of industrial robots, is suitable for industrial automation, intelligent manufacturing and other fields, has broad application prospects, can improve operating efficiency and safety, and optimize the collaboration between robots.
[0066] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A digital twin digital entity modeling method for an industrial robot, characterized in that: The following steps are involved: S1. Obtain the real-time operation data of the physical entity of the robot and the expected joint angle θ c ,The real-time operation data includes the real-time operation speed v and angle θ of the joint, and these data are pre-processed to ensure the validity and consistency of the input data; S2. Build a deep learning model of the robotic arm based on the preprocessed data; S3, combines the deep learning model with the 3D geometric model to build a digital solid model of the robotic arm on the Unity platform; S4, during the operation of the robot arm, updates the data of the digital entity model in real time, and dynamically adjusts and self-improves the model.
2. A digital twin digital entity modeling method for an industrial robot according to claim 1, characterized in that: Step S1 specifically includes: S11, real-time collection of the joint angle θ and speed v operation data of the robot arm; S12, monitoring the environment information around the robot arm, including but not limited to obstacle position P and temperature T parameters; S13, calculate and record the desired joint angle θ of the robot arm through the RRT algorithm c ; S14, collect the actual joint angle θ, speed v, environmental information (P, T) and the expected joint angle θ generated by trajectory planning c The data is transmitted to the controller for preliminary data processing and filtering; S15. The controller stores the processed data in a database for use in subsequent steps of deep learning model training and digital entity model construction.
3. A digital twin digital entity modeling method for an industrial robot according to claim 1, characterized in that: Step S2 specifically includes: S21, the expected joint angle θ preprocessed in step 1 c , joint velocity v and environmental information (P, T) are input into different input layers of the multi-branch neural network respectively; S22. In a multi-branch neural network, an independent sub-network is set for each input to perform feature extraction and analysis; S23, merging the features extracted by each sub-network at the fusion layer of the network to form an overall digital model of the robotic arm; S24. During the model training process, the mean square error (MSE) is used as the loss function. The deviation between the joint angle θ′ predicted by the neural network and the actual observed value θ is calculated, and then the network parameters are adjusted to ensure the accuracy of the model. S25. When the model reaches the preset loss value on the validation set, save the model in onnx format for use in Unity.
4. A digital twin digital entity modeling method for an industrial robot according to claim 3, characterized in that: Step S3 specifically includes: S31. Use the Barracuda plug-in of Unity to import the onnx model trained in step S2 into the Unity environment; S32. Import the URDF model of the robotic arm through URDF Importer and load it into the Unity virtual engine to form a virtual entity robotic arm; S33. In the Unity scene, the actual joint angles, joint velocities, and other input factors required by the model in the Unreal Engine are passed to the Barracuda model; S34, Barracuda model calculates based on input data and outputs predicted actual joint angles; S35. Assign the actual joint angle output by the Barracuda model to the virtual entity in Unity so that it simulates and tracks the motion behavior state of the real robotic arm.
5. A digital twin digital entity modeling method for an industrial robot according to claim 4, characterized in that: Step S4 includes the following contents: S41, using the same method as step S1 to collect the joint angle θ, speed v, and expected joint angle θ′ operation data of the robot arm; and environmental information including the obstacle position P and temperature T around the robot arm; S42. Implement model retraining and fine-tuning, and use new data to incrementally train the model to adapt to changes in the robot's operating environment and performance degradation, ensuring that the digital twin model maintains a high degree of consistency with the physical entity; S43. Through the iterative update mechanism, combined with greedy local search, the adjustable parameters of the model are continuously optimized to achieve consistency measurement between the model and the physical entity, and improve the real-time self-correction capability of the digital twin model; S44. Finally, the updated digital entity model data is stored in the database for subsequent model training, analysis and decision support, forming a closed-loop system of continuous optimization and self-improvement.
Citation Information
Patent Citations
Operation optimization method for long-distance belt conveyor driven by digital twinning
CN114117856A
Industrial mechanical arm control method and system based on digital twin technology
CN115446867A
High-precision mechanical arm control method and system based on reinforcement learning
CN115781685A
Digital twin mechanical arm reinforcement learning training method and system based on Unity
CN117798928A
Industrial robot digital twin model pose precision compensation method, device and system
CN118682764A
