Industrial robot control method and system based on digital twinning

Abstract

The invention discloses an industrial robot control method and system based on digital twinning, and belongs to the technical field of robot control, and the method specifically comprises the steps: collecting the data of an entity industrial robot, building a digital twinning model of a virtual robot and related corollary equipment, and carrying out the digital twinning model of the virtual robot and related corollary equipment in a virtual simulation engine Unity, the method comprises the following steps: configuring a digital twinborn model of a virtual robot and a related environment, performing visual rendering on a scene where the virtual robot is located, displaying a rendered picture on terminal display equipment in real time, obtaining a robot operation instruction, analyzing, synchronously transmitting the operation instruction, and controlling the posture of the robot, according to the method, the physical robot and the virtual robot can be dynamically synchronized by adjusting the physical robot, or the virtual robot and the physical robot can be dynamically synchronized by adjusting the virtual robot, the effect is good, the pertinence is high, the actual operation of each student can be implemented, the safety coefficient is high, and the personal safety cannot be influenced in the real operation.

Description

Technical Field

[0001] The present invention belongs to the technical field of robot control, and specifically relates to an industrial robot control method and system based on digital twins. Background Art

[0002] In recent years, with the rapid development of information technology, the development and production of industrial robots have made great strides. The emergence of various new industrial robots has promoted the development and progress of intelligent manufacturing technology. As an irreplaceable important equipment and means in intelligent manufacturing, industrial robots have played a vital role in industrial production and people's lives.

[0003] With the rapid development of my country's industrial robot industry, the demand for industrial robot technical talents has also increased exponentially, which has led to a shortage of talents in the robot industry. However, there are several common problems with robot teaching in major universities: 1) High cost: Industrial robots are developing rapidly, and universities cannot achieve and meet the replacement of various models of robots; 2) Poor effectiveness and weak pertinence: Considering factors such as time, location, number of students, cost, teaching content, and maintenance, it is almost impossible to implement actual operations for each student; 3) Low safety factor: In actual operations, students' misoperation and other factors will cause equipment loss, especially affecting personal safety. How to effectively implement one-on-one practical operations for each student is the top priority of industrial robot learning.

[0004] For example, a Chinese patent with the authorization announcement number CN115464661B discloses a robot control system based on digital twins, including a data source module, a digital twin module and a control module; the data source module includes a sensor unit and a camera unit; the sensor unit is used to obtain the first posture data of the industrial robot; the camera unit is used to obtain the posture image of the industrial robot; the digital twin module includes a verification unit and a simulation unit; the verification unit is used to obtain the second posture data of the industrial robot and verify whether the first posture data is data with too large an error; the simulation unit is used to control the digital twin model and change the posture of the digital twin model; the control module is used for the staff to control the industrial robot based on the posture of the digital twin model. This invention can effectively reduce the probability that the staff mistakenly stops the operation of the industrial robot because the first posture data is data with too large an error, thereby ensuring the efficiency of industrial production.

[0005] For example, a Chinese patent with authorization announcement number CN115480574B discloses a unit-level digital twin collaborative control method for a mobile robot, which receives given trajectory information, status information of a virtual entity unit, and status information of a physical entity unit; the difference between the given trajectory information and the status information of the virtual entity unit is adjusted by a first backstepping controller to obtain a first control signal; the virtual entity unit is controlled according to the first control signal; the difference between the status information of the physical entity unit and the status information of the virtual entity unit is adjusted by a second backstepping controller to obtain a second control signal; the second control signal and the first control signal are added to obtain a third control signal; the physical entity unit is controlled according to the third control signal; the collaborative control method assists in utilizing the status information of the virtual entity unit to collaboratively control the physical entity unit, and the collaborative control algorithm gives this digital twin system the effect of "controlling the real with the virtual, and reflecting the real with the virtual".

[0006] The defects of the above patents are: 1) Insufficient adaptability to environmental changes, especially in complex or dynamic environments; 2) Sensor noise, delay or data loss will affect the accuracy of the final control signal. Summary of the invention

[0007] In view of the shortcomings of the existing technology, the present invention proposes an industrial robot control method and system based on digital twins, which realizes data synchronization and collaborative control of industrial robots and multiple platforms and multiple data terminals such as PCs, MR mixed reality devices, and mobile tablets, and realizes digital twins and multi-terminal applications of industrial robot application systems.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A digital twin-based industrial robot control method, comprising:

[0010] Collect data of physical industrial robots and build digital twin models of virtual robots and related supporting equipment;

[0011] In the virtual simulation engine Unity, configure the digital twin model of the virtual robot and related environment;

[0012] Visually render the scene where the virtual robot is located, and display the rendered image on the terminal display device in real time;

[0013] Obtain the robot's operating instructions, parse them, transmit them synchronously, and control the robot's posture.

[0014] Specifically, the construction of the digital twin model of the virtual robot and related supporting equipment includes:

[0015] The first model is established based on the homogeneous coordinate transformation matrix. The specific formula is:

[0016] T6=R6(θ1,θ2,θ3)×P6+d6,

[0017] Wherein, T6 represents the transformed six-dimensional coordinate matrix, R6(θ1,θ2,θ3) represents the six-dimensional rotation matrix, θ1,θ2,θ3 represent the rotation angles around the x, y and z axes respectively, P6 represents the position vector in the six-dimensional space, and d6 represents the translation vector in the six-dimensional space;

[0018] The second model is established based on dynamics, and the specific formula is:

[0019]

[0020] Where L represents the power function, q i represents the generalized coordinate value of the ith joint of the physical robot, M i represents the generalized mass matrix, C i represents the Coriolis force matrix, T represents the transpose of a vector or matrix, n represents the number of joints of the physical robot, G i represents the generalized gravity matrix;

[0021] Based on the environment of the physical robot, the third model is established. The specific formula is:

[0022] E E =S E ×P E +N E ,

[0023] Among them, E E represents the environment mapping matrix, S E represents the environmental sensor matrix, P E represents the environment state vector, N E represents the noise matrix;

[0024] The first model, the second model and the third model are integrated to construct a digital twin model of the physical robot.

[0025] Specifically, the physical industrial robot data includes:

[0026] The names, dimensions, weights, and relationships between the parts of a physical industrial robot. The parts include the robot body base, waist, upper arm, lower arm, wrist, and flange.

[0027] Specifically, the obtaining and parsing of the robot operation instructions, synchronous transmission of the operation instructions, and controlling the posture of the robot include:

[0028] Obtain robot operation instructions, and parse the obtained robot operation instructions to obtain a multi-dimensional control signal vector C0 suitable for the robot;

[0029] The robot’s multi-dimensional control signal vector is transmitted to the robot through the 5G network, and the delay in the signal transmission process is compensated to obtain the robot’s multi-dimensional control signal vector C after delay compensation. s ;

[0030] According to the real-time robot multi-dimensional control signal vector, the robot's posture is adjusted, the robot's posture is monitored in real time, and the control parameters are adjusted. The specific formula is:

[0031] A C =J C ×q C +Kz C (Zt C -Zt R ),

[0032] Among them, A C represents the robot posture adjustment vector, J C represents the Jacobian matrix of the robot posture control, q C Represents the robot posture control velocity vector, Kz C represents the control gain matrix, Zt C represents the robot target posture matrix, Zt R Represents the robot's real-time posture feedback matrix.

[0033] Specifically, the obtaining of robot operation instructions, parsing, synchronous transmission of operation instructions, and controlling the posture of the robot also include:

[0034] The physical robot and the virtual robot are dynamically synchronized and adjusted in real time through the digital twin model. During the dynamic synchronization process, the posture error between the physical robot and the virtual robot is calculated and corrected. The specific formula is:

[0035] ΔE=Zt st -Zt xn , T S =Zt R +K E ×ΔE,

[0036] Among them, △E represents the attitude error matrix, Zt st Represents the posture matrix of the physical robot, Zt Xn represents the posture matrix of the virtual robot, T s represents the posture matrix after the virtual robot and the physical robot are synchronized, K E represents the error correction gain matrix.

[0037] Specifically, the terminal display device includes: a personal computer, an MR mixed reality device, a mobile phone, a tablet and a client.

[0038] An industrial robot control system based on digital twins, used to implement the industrial robot control method based on digital twins, comprising: a data acquisition module, a model configuration module, a rendering display module and a posture control module;

[0039] The data acquisition module is used to collect data of the physical industrial robot and build a digital twin model of the virtual robot and related supporting equipment;

[0040] The model configuration module is used to configure the digital twin model and related environment of the virtual robot in the virtual simulation engine Unity;

[0041] The rendering and display module is used to perform visual rendering of the scene where the virtual robot is located, and to display the rendered image on the terminal display device in real time;

[0042] The posture control module is used to obtain and analyze the robot operation instructions, synchronously transmit the operation instructions, and control the robot's posture.

[0043] Specifically, the posture control module includes: an instruction parsing unit, a delay compensation unit, a posture control unit and an error adjustment unit;

[0044] The instruction parsing unit is used to obtain the robot operation instruction and parse the obtained robot operation instruction;

[0045] The delay compensation unit is used to transmit the multi-dimensional control signal vector of the robot to the robot through the 5G network and compensate for the delay in the signal transmission process;

[0046] The posture control unit is used to adjust the posture of the robot according to the real-time multi-dimensional control signal vector of the robot, monitor the posture of the robot in real time, and adjust the control parameters;

[0047] The error adjustment unit is used to calculate and correct the posture error between the physical robot and the virtual robot during the dynamic synchronization process between the physical robot and the virtual robot.

[0048] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of an industrial robot control method based on digital twins when executing the computer program.

[0049] A computer-readable storage medium stores computer instructions, which, when executed, execute the steps of an industrial robot control method based on digital twins.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] 1. The present invention proposes an industrial robot control method based on digital twins. By constructing a digital twin model, the operating status of the industrial robot can be monitored in real time to achieve accurate data collection and analysis.

[0052] 2. The present invention proposes an industrial robot control method based on digital twins. By synchronously controlling the physical robot and the digital twin model, the robot's motion accuracy under different operating conditions can be ensured. Especially in high-precision tasks, this synchronous control can greatly improve the quality and consistency of the product.

[0053] 3. The present invention proposes an industrial robot control method based on digital twins, which is effective and targeted, can implement the actual operation of each student, has a high safety factor, and will not affect personal safety in actual operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A flow chart of an industrial robot control method based on digital twin provided by the present invention;

[0055] Figure 2 A complete flow chart for establishing a virtual model for the physical industrial robot and supporting equipment provided by the present invention;

[0056] Figure 3 A comparison diagram of the physical industrial robot device provided by the present invention and the modeled virtual industrial robot model;

[0057] Figure 4 Socket communication model flow chart provided by the present invention;

[0058] Figure 5 An architecture diagram of an industrial robot control system based on digital twins provided by the present invention;

[0059] Figure 6 A schematic diagram of an electronic device provided by the present invention;

[0060] Figure 7 A schematic diagram of the computer-readable storage medium structure provided by the present invention. DETAILED DESCRIPTION

[0061] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, in the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate the orientation or position relationship based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "No. 1", "No. 2" and "No. 3" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. The present invention is further explained below in conjunction with specific embodiments.

[0062] Example 1

[0063] See also Figure 1-Figure 4 , an embodiment provided by the present invention: an industrial robot control method based on digital twin, comprising the following specific steps:

[0064] Step S1: Collect data of the physical industrial robot and build a digital twin model of the virtual robot and related supporting equipment;

[0065] The specific steps of step S1 include:

[0066] Step S101: Establish a first model based on the homogeneous coordinate transformation matrix. The specific formula is:

[0067] T6=R6(θ1,θ2,θ3)×P6+d6,

[0068] Among them, T6 represents the transformed six-dimensional coordinate matrix, that is, the final position and momentum state of the physical robot in the six-dimensional space, R6(θ1,θ2,θ3) represents the six-dimensional rotation matrix, that is, the three independent rotations of the physical robot in the three-dimensional space. Through the compound rotation operation, the physical robot is transformed from the original state to the target state. θ1,θ2,θ3 represent the rotation angles around the x, y and z axes respectively, defining the rotation of the physical robot in space, R x (θ1), R y (θ2) and R z (θ3) represents the uniaxial rotation matrix around the x, y and z axes respectively, and P6 represents the position vector in the six-dimensional space, including the three-dimensional position (x, y, z) and the three-dimensional momentum component (p x ,p y ,p z ), used to describe the physical state of the physical robot, d6 represents the translation vector in the six-dimensional space, that is, the linear displacement of the object from the initial position to the target position;

[0069] Ry (θ2), R z (θ3) and R x The format of (θ1) is similar and constructed in the same way.

[0070] The principle of the first model: It is mainly used to describe the position and momentum state of the industrial robot in the six-dimensional space. The precise positioning and direction adjustment of the robot in space can be achieved through the rotation matrix R6 (θ1, θ2, θ3). This rotation operation combines three independent rotation angles θ1, θ2, θ3, allowing the robot to move flexibly in a complex environment. The position vector P6 contains the physical state of the robot in the six-dimensional space, and the translation vector d6 is used to achieve linear displacement in space.

[0071] Step S102: Establish a second model based on dynamics, the specific formula is:

[0072]

[0073] Where L represents the power function, that is, the difference between the generalized kinetic energy and potential energy of the physical robot, q i Represents the generalized coordinate value of the ith joint of the physical robot, including the displacement, velocity and other parameters of the joint, which is used to describe the motion state of the multi-degree-of-freedom robot. i represents the generalized mass matrix, which describes the inertia, mass and coupling relationship between the i-th joint, C i represents the Coriolis force matrix, that is, the coupling effect of the generalized force generated by the movement between joints, reflecting the nonlinear characteristics of the system, T represents the transpose of the vector or matrix, n represents the number of joints of the physical robot, G i represents the generalized gravity matrix, which takes into account the influence of gravity on the system.

[0074] The principle of the second model: The dynamic model based on Lagrangian mechanics is used to describe the motion equations of multi-degree-of-freedom industrial robots. The power function L in the formula represents the difference between the generalized kinetic energy and potential energy of the system. i and the generalized mass matrix M i , which can describe the motion state and inertial characteristics of each joint, the Coriolis force matrix C i It is used to deal with the nonlinear coupling effect between joints. The generalized gravity matrix G i The effect of gravity on the system is taken into account.

[0075] Step S103: Based on the environment where the physical robot is located, a third model is established. The specific formula is:

[0076] E E =S E ×P E +NE ,

[0077] Among them, E E Represents the environment mapping matrix, that is, the environment state of the physical robot, S E represents the environmental sensor matrix, i.e., the environmental information collected by the sensor, P E Represents the environmental state vector, which contains information such as temperature, humidity, obstacle location, etc. E represents the noise matrix, which is used to process the noise in the sensor data;

[0078] The principle of the third model: The environment mapping formula is used to map the characteristics of the physical robot's environment (such as temperature, humidity, obstacle location, etc.) to the digital twin model. E Represents the environmental sensor matrix, which contains the environmental information collected by the sensor. The environmental data captured by the sensor is expressed through the environmental state vector P E Indicates that the vector contains environmental parameters such as temperature, humidity, light, and obstacle position, and the noise matrix N E Used to handle noise in sensor data and ensure accuracy during mapping.

[0079] The core idea of ​​this formula is to integrate the data of the environmental sensor into the environmental mapping matrix E through linear transformation. E , this matrix represents the state of the environment in the digital twin model, and the digital twin model can be based on E E The robot can respond and adjust to the environmental status reflected in real time to ensure that it can adapt to different working environments.

[0080] Step S104: Integrate the first model, the second model and the third model to construct a digital twin model of the physical robot.

[0081] T DT =T6+L+E E , T DT Represents the final digital twin model mapping matrix, which contains information of all dimensions for real-time control and monitoring.

[0082] Before developing virtual simulation software, it is necessary to make a model. First, the IRB120 physical industrial robot application system is selected as the research object, and three-dimensional modeling is performed on it. Equipment information collection is the first step in model making. In the process of collecting equipment information, it is necessary to focus on collecting detailed information of the equipment, such as the size of the tool, the distribution of the operation module, etc., for subsequent proportional modeling based on the equipment information. At the same time, the images collected on the spot can be used as model maps to increase the sense of reality.

[0083] The data of physical industrial robots include: the names, dimensions, weights, and relationships between the parts of the physical industrial robot. These parts include the robot body base, waist, upper arm, lower arm, wrist, flange, etc.

[0084] The names, dimensions, weights and dependencies of the components of the supporting equipment surrounding the physical industrial robot. These components include the execution structures and fixtures of the storage station, grinding station, palletizing station, visual station and assembly station;

[0085] These data are recorded in real time to the PLC data storage device through the data port; based on the above-mentioned component names, dimensions, weights, affiliations and other data, a virtual model and scene of the physical robot are constructed in the design simulation software NX, wherein the virtual model is composed of multiple virtual components, and the names, dimensions and other parameters of the virtual components correspond one-to-one with the names and dimensions of the components of the physical industrial robot. The connection method between the virtual components, the affiliation of the joints in the power chain, etc., also correspond one-to-one with the components on the physical industrial robot; simulation is performed based on the virtual model composed of the various virtual components to obtain a real-time dynamic virtual model of the industrial robot.

[0086] Step S2: In the virtual simulation engine Unity, the digital twin model of the virtual robot and the related environment are configured, including the setting of the rotation axis, joint linkage relationship, and related physical properties of supporting facilities;

[0087] The embodiment of the present invention collects information about the ABBIRB120 industrial robot based on the engineering data model provided by the manufacturer, combined with physical measurements and instructions, and determines the structure and specific dimensions of the industrial robot application system after aggregating multiple data. These data include both the ABBIRB120 industrial robot body data and the information of its supporting facilities and process flow parts;

[0088] Since the physical robot and supporting facilities used in the examples of the present invention have complex structures and numerous components, the powerful and easy-to-operate NX software is used for modeling. During the modeling process, the size of the model is determined according to the size of the actual equipment, such as Figure 2 , a complete process for building virtual models of physical industrial robots and supporting equipment.

[0089] After completing the 3D modeling of the robot and supporting facilities, the model is converted. The 3D model is saved in .step format in NX software, converted to .fbx format by DeepExploration, and finally imported into the virtual reality engine Unity3D in .fbx format. After the model is imported into Unity3D, there are often problems such as redundant information and too many triangular faces of the mesh model. It is necessary to optimize it in NX in advance to reduce the triangular faces of the industrial robot part model. Figure 3 The comparison between the physical industrial robot equipment and the modeled virtual industrial robot model is schematically shown.

[0090] While completing the modeling, configuration is performed through TIA Portal software to complete the OPC connection and PLC configuration, and collect industrial robot related data in the PLC.

[0091] OPC (OLE for Process Control) is OLE (Object Linking and Embedding) for process control. OLE object linking and embedding is a basic early building block of Windows, allowing applications to share complex information between them. OPC UA (OPC Unified Architecture) is the next generation of OPC standards, providing a complete, secure and reliable cross-platform architecture to obtain real-time and historical data and time.

[0092] The API (Application Programming Interface) in this system is composed of a set of definitions, programs and protocols, and the API interface enables mutual communication between computer software. The main function of the API is to provide a reusable set of common functions. OPC UA uses these APIs to establish a client / server model for communication and data collection. The server defines the address space and provides an interface to the outside world. Authorized clients call services through the API to communicate with the server, browse the address space, read, write and subscribe to data.

[0093] In this way, the data of the physical industrial robot is collected into the data memory of the PLC via Profinet communication. The server can access the data block memory of the PLC by configuring it through the OPC server in the TIAPortal software;

[0094] OPC server data access includes publish-subscribe mode. The specific process of subscription is that the client sends a subscription request to the server, the server responds to create monitoring items, and then returns the required data information. The OPCXML module exports data into the XML format based on the OPC Foundation. The import and export of XML format data can be integrated into the configuration tool in a one-click manner. After completing the configuration, in subsequent data access, you only need to open the OPC server to automatically refresh the data to the server's XML data pool in real time.

[0095] Since XML has a unified standard syntax and can easily support the use of any system and product, the embodiment of the present invention creates an information model based on the industrial robot information obtained from the XML data pool.

[0096] Step S3: The display terminal held by the user performs visual rendering of the scene where the virtual robot is located according to the position and observation angle of the user (observer), and displays the rendered image on the terminal display device in real time;

[0097] The display terminals held by the users include: personal computers, MR mixed reality devices, mobile phones, tablets and other multi-platforms, multi-clients, etc. These systems connect to the server through the Socket communication port through the virtual simulation program to obtain data distributed by the 5G data network.

[0098] Socket is an intermediate software abstraction layer for communication between the application layer and the TCP / IP protocol. It is a set of interfaces. For users, Socket organizes data to comply with the specified protocol. Using Socket to read OPC real-time data is a common method of cross-platform real-time data exchange. It requires real-time and integrity of data collection and transmission, and can solve the problems caused by operating systems and firewalls to OPC users. The Unity3D development environment also supports Socket communication. The Socket client is responsible for applying for connection, and the server is responsible for completing the connection through application. The Socket communication model is as follows: Figure 4 shown.

[0099] Step S4: Obtaining the robot operation instructions, parsing them, transmitting the operation instructions synchronously, and controlling the robot's posture;

[0100] The specific steps of step S4 are:

[0101] Step S401: Obtain robot operation instructions, and parse the obtained robot operation instructions to obtain a multi-dimensional control signal vector suitable for the robot, including parameters such as position, posture, speed and acceleration, and set the robot's multi-dimensional control signal vector as C0. Among them, P0 represents the initial position vector of the robot, V0 represents the velocity vector of the robot, A0 represents the acceleration vector of the robot, R0(θ) represents the posture rotation matrix of the robot, that is, the angle change, and t represents time;

[0102] The robot operation instructions in step S401 include: physical robot operation instructions or virtual robot operation instructions.

[0103] Step S402: Transmit the robot's multi-dimensional control signal vector to the robot through the 5G network, and compensate for the delay in the signal transmission process. The specific formula is:

[0104] C s =C0+D B (Δt),

[0105] Among them, C s represents the multi-dimensional control signal vector of the robot after delay compensation, D B (Δt) represents a delay compensation function, which is adjusted for the time delay Δt, where Δt represents the time delay, i.e., the time difference generated during the transmission of the control signal vector;

[0106] D B (Δt) = K B ×Δt×exp(-λ×Δt)+G B sin(ω×Δt),

[0107] Among them, K B represents the time gain matrix, which is used to adjust the influence of time delay on the control signal vector. λ represents the attenuation coefficient, which controls the exponential attenuation speed of the compensation signal with time delay to avoid overcompensation. B represents the frequency gain matrix, which is used to adjust the frequency response of delay compensation. ω represents the angular frequency, that is, the oscillation frequency of the delay compensation function, to ensure the smoothness of the dynamic response.

[0108] The principle of the delay compensation function: The exp(-λ×Δt) part introduces an exponential decay term to prevent the compensation signal from being over-enhanced as the time delay increases. This decay mechanism ensures the stability of the compensation and avoids system oscillation. The sinusoidal term sin(ω×Δt) introduces a dynamic adjustment mechanism for delay compensation to ensure that the compensation signal can flexibly respond to different time delays, especially providing a smooth compensation effect in high-frequency oscillation control systems.

[0109] Step S403: According to the real-time robot multi-dimensional control signal vector, the robot's posture is adjusted, the robot's posture is monitored in real time, and the control parameters are adjusted. The specific formula is:

[0110] A C =J C ×q C +Kz C (Zt C -Zt R ),

[0111] Among them, A C represents the robot posture adjustment vector, J C represents the robot posture control Jacobian matrix, q C Represents the robot posture control velocity vector, Kz C represents the control gain matrix, Zt C represents the robot target posture matrix, Zt R Represents the robot's real-time posture feedback matrix.

[0112] The principle of adjusting the control parameter formula, A C Represents the adjustment signal of the robot's posture. This vector determines the amount of change in the robot's posture after receiving the operation command. By integrating the difference in posture control speed and real-time feedback, A C Ensure that the robot can adjust its posture in real time and maintain operational accuracy. C It is used to link the robot joint velocity with the posture change of the end effector. This matrix is ​​the core of posture control. It describes the influence of the movement of each joint of the robot on the final posture. C , can accurately control the robot's posture changes, q C Represents the instantaneous motion speed of each joint of the robot. This vector determines the speed of the robot's posture change under a given operation instruction. By combining J C , achieving real-time adjustment of posture, Kz C It is used to adjust the influence of feedback signals in the process of attitude control. The size of this matrix directly affects the response speed and stability of the system. By adjusting Kz C , which enables more precise posture control and avoids over- or under-adjustment.

[0113] Step S404: Dynamically synchronize the physical robot and the virtual robot, and make real-time adjustments through the digital twin model. During the dynamic synchronization process, the posture error between the physical robot and the virtual robot is calculated and corrected. The specific formula is:

[0114] ΔE=Zt st -Zt xn , T S =Zt R +K E ×ΔE,

[0115] Among them, △E represents the posture error matrix, that is, the posture error between the virtual robot and the physical robot, Zt st Represents the posture matrix of the physical robot, Zt Xn represents the posture matrix of the virtual robot, T s represents the posture matrix after the virtual robot and the physical robot are synchronized, K E represents the error correction gain matrix.

[0116] The principle of the above formula: Through complex mathematical models and real-time feedback mechanisms, the accurate transmission of operating instructions and posture control are ensured, and a high degree of synchronization between virtual and physical robots is achieved. The posture error correction mechanism improves the synchronization accuracy, allowing digital twin technology to be better applied to industrial robot control.

[0117] In this embodiment, posture control is combined with delay compensation and error correction to ensure high-precision operation of the robot in complex environments. The dynamic synchronization mechanism ensures real-time consistency between the virtual and physical robots. In dynamic synchronization, the virtual robot can be dynamically synchronized by adjusting the physical robot, and the physical robot can also be dynamically synchronized by adjusting the virtual robot.

[0118] Example 2

[0119] See also Figure 5 , another embodiment provided by the present invention: an industrial robot control system based on digital twins, comprising: a data acquisition module, a model configuration module, a rendering and display module and a posture control module;

[0120] The data acquisition module is used to collect data of the physical industrial robot and build a digital twin model of the virtual robot and related supporting equipment;

[0121] The model configuration module is used to configure the digital twin model and related environment of the virtual robot in the virtual simulation engine Unity;

[0122] The rendering and display module is used to perform visual rendering of the scene where the virtual robot is located, and to display the rendered image on the terminal display device in real time;

[0123] The posture control module is used to obtain and analyze the robot operation instructions, synchronously transmit the operation instructions, and control the robot's posture.

[0124] The attitude control module includes: an instruction parsing unit, a delay compensation unit, an attitude control unit and an error adjustment unit;

[0125] The instruction parsing unit is used to obtain the robot operation instruction and parse the obtained robot operation instruction;

[0126] The delay compensation unit is used to transmit the multi-dimensional control signal vector of the robot to the robot through the 5G network and compensate for the delay in the signal transmission process;

[0127] The posture control unit is used to adjust the posture of the robot according to the real-time multi-dimensional control signal vector of the robot, monitor the posture of the robot in real time, and adjust the control parameters;

[0128] The error adjustment unit is used to calculate and correct the posture error between the physical robot and the virtual robot during the dynamic synchronization process between the physical robot and the virtual robot.

[0129] Example 3

[0130] Figure 61 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 6 As shown, according to another aspect of the present application, an electronic device 500 is also provided. The electronic device 500 may include one or more processors and one or more memories. Among them, the memory stores a computer readable code, and when the computer readable code is run by one or more processors, a method for controlling an industrial robot based on digital twins can be executed.

[0131] The method or system according to the embodiment of the present application can also be used by Figure 6 The electronic device architecture shown in FIG. Figure 6 As shown, the electronic device 500 may include a bus 501, one or more CPUs 502, ROM 503, RAM 504, a communication port 505 connected to a network, an input / output component 506, a hard disk 507, and the like. The storage device in the electronic device 500, such as ROM 503 or hard disk 507, may store an industrial robot control method based on digital twins provided in the present application. An industrial robot control method based on digital twins may, for example, include: collecting data of a physical industrial robot, building a digital twin model of a virtual robot and related supporting equipment, configuring the digital twin model of the virtual robot and the related environment in the virtual simulation engine Unity, visually rendering the scene where the virtual robot is located, and displaying the rendered image in real time on the terminal display device, obtaining robot operation instructions, parsing them, synchronously transmitting the operation instructions, and controlling the posture of the robot. Furthermore, the electronic device 500 may also include a user interface 508. Of course, Figure 6 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 6 One or more components of an electronic device are shown.

[0132] Example 4

[0133] Figure 7 Schematic diagram of a computer-readable storage medium structure provided by an embodiment of the present application. Figure 7 As shown, it is a computer-readable storage medium 600 according to an embodiment of the present application. Computer-readable instructions are stored on the computer-readable storage medium 600. When the computer-readable instructions are executed by the processor, a method for controlling an industrial robot based on digital twins according to an embodiment of the present application described with reference to the above figures can be executed. The storage medium 600 includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0134] It should be understood that the methods, apparatuses, and devices of the present application can be implemented in many ways. For example, the methods, apparatuses, and devices of the present application can be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is only for illustration, and the steps of the method of the present application are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present application can also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers a recording medium storing a program for executing the method according to the present application.

[0135] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0136] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A digital twin-based industrial robot control method, characterized in that: include: Collect data of physical industrial robots and build digital twin models of virtual robots and related supporting equipment; In the virtual simulation engine Unity, configure the digital twin model of the virtual robot and related environment; Visually render the scene where the virtual robot is located, and display the rendered image on the terminal display device in real time; Obtain the robot's operating instructions, parse them, transmit them synchronously, and control the robot's posture.

2. The industrial robot control method based on digital twin according to claim 1, characterized in that: The construction of the digital twin model of the virtual robot and related supporting equipment includes: The first model is established based on the homogeneous coordinate transformation matrix. The specific formula is: T6=R6(θ1,θ2,θ3)×P6+d6, Wherein, T6 represents the transformed six-dimensional coordinate matrix, R6(θ1,θ2,θ3) represents the six-dimensional rotation matrix, θ1,θ2,θ3 represent the rotation angles around the x, y and z axes respectively, P6 represents the position vector in the six-dimensional space, and d6 represents the translation vector in the six-dimensional space; The second model is established based on dynamics, and the specific formula is: Where L represents the power function, q i represents the generalized coordinate value of the ith joint of the physical robot, M i represents the generalized mass matrix, C i represents the Coriolis force matrix, T represents the transpose of a vector or matrix, n represents the number of joints of the physical robot, G i represents the generalized gravity matrix; Based on the environment of the physical robot, the third model is established. The specific formula is: E E =S E ×P E +N E Among them, E E represents the environment mapping matrix, S E represents the environmental sensor matrix, P E represents the environment state vector, N E represents the noise matrix; The first model, the second model and the third model are integrated to construct a digital twin model of the physical robot.

3. The industrial robot control method based on digital twin according to claim 2, characterized in that: The physical industrial robot data includes: The names, dimensions, weights, and relationships between the parts of a physical industrial robot. The parts include the robot body base, waist, upper arm, lower arm, wrist, and flange.

4. The industrial robot control method based on digital twin according to claim 3, characterized in that: The obtaining and parsing of the robot operation instructions, synchronous transmission of the operation instructions, and control of the robot posture include: Obtain robot operation instructions, and parse the obtained robot operation instructions to obtain a multi-dimensional control signal vector C0 suitable for the robot; The robot’s multi-dimensional control signal vector is transmitted to the robot through the 5G network, and the delay in the signal transmission process is compensated to obtain the robot’s multi-dimensional control signal vector C after delay compensation. s ; According to the real-time robot multi-dimensional control signal vector, the robot's posture is adjusted, the robot's posture is monitored in real time, and the control parameters are adjusted. The specific formula is: WHOSE C =J C ×q C +Kz C (Zt C -Zt R ), Among them, A C represents the robot posture adjustment vector, J C represents the robot posture control Jacobian matrix, q C Represents the robot posture control velocity vector, Kz C represents the control gain matrix, Zt C represents the robot target posture matrix, Zt R Represents the robot's real-time posture feedback matrix.

5. The industrial robot control method based on digital twin according to claim 4, characterized in that: The obtaining and parsing of the robot operation instructions, synchronous transmission of the operation instructions, and controlling the posture of the robot also include: The physical robot and the virtual robot are dynamically synchronized and adjusted in real time through the digital twin model. During the dynamic synchronization process, the posture error between the physical robot and the virtual robot is calculated and corrected. The specific formula is: ΔE=Zt st -Zt xn ,T S =Zt R +K E ×ΔE, Among them, △E represents the attitude error matrix, Zt st Represents the posture matrix of the physical robot, Zt Xn represents the posture matrix of the virtual robot, T s represents the posture matrix after the virtual robot and the physical robot are synchronized, K E represents the error correction gain matrix.

6. The industrial robot control method based on digital twin according to claim 5, characterized in that: The terminal display device includes: a personal computer, an MR mixed reality device, a mobile phone, a tablet and a client.

7. An industrial robot control system based on digital twin, used to implement an industrial robot control method based on digital twin according to any one of claims 1 to 6, characterized in that: include: Data acquisition module, model configuration module, rendering display module and attitude control module; The data acquisition module is used to collect data of the physical industrial robot and build a digital twin model of the virtual robot and related supporting equipment; The model configuration module is used to configure the digital twin model and related environment of the virtual robot in the virtual simulation engine Unity; The rendering and display module is used to perform visual rendering of the scene where the virtual robot is located, and to display the rendered image on the terminal display device in real time; The posture control module is used to obtain and analyze the robot operation instructions, synchronously transmit the operation instructions, and control the robot's posture.

8. The industrial robot control system based on digital twin according to claim 7, characterized in that: The posture control module includes: an instruction parsing unit, a delay compensation unit, a posture control unit and an error adjustment unit; The instruction parsing unit is used to obtain the robot operation instruction and parse the obtained robot operation instruction; The delay compensation unit is used to transmit the multi-dimensional control signal vector of the robot to the robot through the 5G network and compensate for the delay in the signal transmission process; The posture control unit is used to adjust the posture of the robot according to the real-time multi-dimensional control signal vector of the robot, monitor the posture of the robot in real time, and adjust the control parameters; The error adjustment unit is used to calculate and correct the posture error between the physical robot and the virtual robot during the dynamic synchronization process between the physical robot and the virtual robot.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of an industrial robot control method based on digital twins described in any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed, the steps of an industrial robot control method based on digital twins described in any one of claims 1 to 6 are executed.

Images