Polishing process virtualization simulation system and method based on Unity 3D

Abstract

The invention discloses a polishing process virtualization simulation system and method based on Unity3D, and relates to the field of virtualization simulation. Polishing process virtualization is implemented based on a Unity3D platform; the system not only provides a safe and efficient virtual training platform for operators, but also provides firm technical support for teleoperation through an accurate master-slave equipment space mapping technology; by combining a Geomagic Touch force feedback device and an HTC VIVE Pro 2 virtual reality device, the embodiment of the invention realizes accurate space mapping between heterogeneous models, and constructs a vivid virtual polishing environment; precise mechanical arm control is achieved by designing a space mapping algorithm between heterogeneous models, a mesh model of a workpiece is generated through a mesh subdivision method, increasing and distributing vertex positions and constructing a triangular face, the grinding function is achieved based on a Mesh cutting algorithm, the force effect in the grinding process is provided through force feedback equipment, and the grinding precision is improved. And the purpose of polishing process virtualization is achieved.

Description

Technical Field

[0001] The present invention belongs to the field of virtualization simulation, and in particular, relates to a polishing process virtualization simulation system and method based on Unity3D. Background Art

[0002] In the process of intelligent manufacturing, some high-skill and high-risk process operations are still difficult for non-technical personnel to master; these operations not only have a direct impact on production efficiency and product quality, but also pose a challenge to the safety of practice; in this context, the development of processing application programs for industrial robots has become a hot topic; especially the grinding process, which plays an important role in improving the surface quality of workpieces, but also faces problems such as difficult quality control, low efficiency and high cost; although the virtualization of the grinding process faces challenges such as accurate simulation of physical effects, accurate reproduction of material properties, improved authenticity of interactive experience and enhanced user adaptability, its importance cannot be underestimated; on the one hand, it can rehearse the grinding process in a virtual environment, serve as an efficient training platform, and reduce operational risks; on the other hand, it can serve as a solid support platform for remote operation, improve the accuracy and stability of remote operation through virtual simulation, and promote the development of remote operation technology; existing virtual training and remote operation research have improved the safety of operation to a certain extent, but lack high-fidelity visual and tactile interaction. Summary of the invention

[0003] In view of the problems in the related art, the present invention proposes a polishing process virtualization simulation system and method based on Unity3D to overcome the above-mentioned technical problems existing in the existing related technology.

[0004] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0005] The present invention is a grinding process virtualization simulation method based on Unity3D, comprising the following steps:

[0006] S1. Establish a model and convert it to construct a polishing robot model suitable for Unity3D, and obtain the final polishing robot model; set the master device and the slave device;

[0007] S2. Define the working space range of the master device and the slave device to obtain the working space range of the master end and the working space range of the slave end;

[0008] S3, setting a mapping function and mapping function parameters according to the difference between the working space range of the master end and the working space range of the slave end; then obtaining each joint angle of the master end device in real time and using forward kinematics to calculate its posture to obtain posture data; then inputting the posture data into the mapping function to calculate the target posture of the slave end; and using inverse kinematics to calculate the rotation angle of each joint of the slave end device in conjunction with the target posture of the slave end;

[0009] S4, realize the virtualization of the polishing process by cooperating with the final polishing robot arm model, posture data and the rotation angles of each joint of the slave device;

[0010] Preferably, the master device in S1 includes Geomagic Touch, and the slave device includes an RM65-B robotic arm;

[0011] Preferably, the S1 comprises the following steps:

[0012] S11. Use SolidWorks software to accurately model the virtual grinding robot arm to obtain an initial robot arm model;

[0013] S12, using Blender software to convert the right-hand coordinate system of the initial mechanical arm model into the left-hand coordinate system required by Unity3D, and placing the initial mechanical arm model at the origin to eliminate the offset, thereby obtaining a converted mechanical arm model;

[0014] S13. According to the specific structure of the robotic arm, a mesh collider, a rigid body component and a material are added to the converted robotic arm model to obtain a final polished robotic arm model;

[0015] In the virtual simulation system, model building and tactile feedback are two core elements to achieve a realistic operating experience. In order to ensure the accuracy and functionality of the model in Unity3D, the virtual polishing robot arm is firstly accurately modeled using SolidWorks software. Secondly, in order to solve the difference in coordinate systems between different software, Blender software is used to convert the right-hand coordinate system of the model into the left-hand coordinate system required by Unity3D, and the model is placed at the origin to eliminate the offset. Thirdly, the robot components are organized by establishing a parent-child relationship, so that the child object inherits the parent object transformation and forms an ordered hierarchical structure to facilitate complex motion and structural organization. Finally, in order to achieve accurate collision detection and tactile feedback, mesh colliders, rigid body components and materials are added to the robot arm according to its specific structure. At the same time, in order to simulate a real tactile interaction scenario, rigid bodies, colliders and materials are also added to the workpiece.

[0016] Preferably, S2 comprises the following steps:

[0017] S21, using the Monte Carlo method and combining the DH parameters of the master device and the slave device to solve the working space of the master device and the slave device, and obtain the working space diagram of the master device and the slave device;

[0018] Preferably, setting the mapping function according to the difference between the master end working space range and the slave end working space range in S3 comprises the following steps:

[0019] S31, listing a table of working space ranges of the master-end device and the slave-end device according to the working space diagrams of the master-end device and the slave-end device;

[0020] S32, constructing a mapping function according to the working space range table of the master device and the slave device; as follows,

[0021] S = KL + B;

[0022] In the formula, S represents the end position vector of the master device, L represents the end target position vector of the robot, K represents the spatial mapping coefficient, and B represents the constant that the origin position of the master device workspace after proportional enlargement is consistent with the origin position of the robot workspace;

[0023] Preferably, S4 comprises the following steps:

[0024] S41, performing heterogeneous control, polishing function development and VR interaction to build a virtual scene in cooperation with the final polishing robot arm model, posture data and the rotation angles of each joint of the slave device, to obtain a built virtual scene;

[0025] S42, performing a grinding operation simulation operation in the constructed virtual scene to realize grinding process virtualization;

[0026] Preferably, performing heterogeneous control in S41 includes the following steps:

[0027] S4111. Position and attitude are usually used to determine the state of the end of the robot in three-dimensional space;

[0028] S4112, converting the left-hand coordinate system into the right-hand coordinate system according to the state of the end of the robotic arm in the three-dimensional space;

[0029] S4113, real-time acquisition of the joint angle data of the master-end device, and the preparation of a forward kinematics script to calculate the terminal posture in conjunction with the final polishing robot arm model, posture data, and the rotation angles of each joint of the slave-end device, and the posture is converted into posture data that can be recognized by the robot arm, and finally the heterogeneous model control script is used to realize precise control of the master and slave ends;

[0030] The pose obtained by Unity3D's Transform component is in the left-hand coordinate system, but the kinematic modeling of the robotic arm usually uses the right-hand coordinate system; in order to ensure the control accuracy of the master and slave ends, the left-hand coordinate system must be converted to the right-hand coordinate system.

[0031] Preferably, the S4112 includes the following steps:

[0032] S41121, let the displacement along the X, Y, and Z axes in the left-hand system be P XL , P YL , P ZL , the rotation angle is θ L ,φ L , The corresponding displacement of the right-hand system is P XR , P YR , P ZR , the rotation angles are θ R ,φ R , From this, we can see that there is the following mathematical mapping relationship between the left-handed coordinate system posture and the right-handed coordinate system posture;

[0033]

[0034] Preferably, the polishing function development in S41 includes the following steps:

[0035] S4121. First, set the attributes of the model and the number of mesh subdivisions. Use the mesh subdivision method to add vertices to create the model mesh, calculate the position of each vertex in the scene, set the vertex coordinate position array of each face of the workpiece based on the local coordinates of the vertex, and then set the index array (Triangles) of the triangles of each face of the workpiece according to the vertex subscript, dynamically create a mesh model of the workpiece, and add material to it.

[0036] S4122, create a grinding head, convert the world coordinates of the grinding head into local coordinates, loop through all vertex arrays of the workpiece vertices, first determine whether the y value of the vertex coordinate of the grinding head is less than the y value of the vertex coordinate of the workpiece, then screen whether the vertex coordinate of the workpiece is within the radius range of the grinding head in the two-dimensional plane, if it is within the radius range, continue to screen all vertices within the radius range in the three-dimensional plane, if it is within this range, dynamically modify the y value of the vertex coordinate of the workpiece to the y value of the local coordinate of the grinding head, thereby simulating material removal, and finally update the vertex information, recalculate the normal and tangent of the workpiece mesh, and update the Mesh rendering to realize the grinding function;

[0037] A polishing process virtualization simulation system based on Unity3D, including a model building conversion module, a workspace scope definition module, a master-slave end mapping building module and a polishing process virtualization module;

[0038] The model building conversion module is used to build and convert the model, build a polishing robot model suitable for Unity3D, obtain the final polishing robot model and set the master end device and the slave end device;

[0039] The working space range defining module is used to define the working space ranges of the master end device and the slave end device, and obtain the master end working space range and the slave end working space range;

[0040] The master-slave mapping construction module is used to set a mapping function and mapping function parameters according to the difference between the master workspace range and the slave workspace range; then obtain each joint angle of the master device in real time and use forward kinematics to calculate its posture to obtain posture data; then input the posture data into the mapping function to calculate the target posture of the slave; and use inverse kinematics to calculate the rotation angle of each joint of the slave device in conjunction with the target posture of the slave;

[0041] The polishing process virtualization module is used to realize the virtualization of the polishing process in conjunction with the final polishing robot arm model, posture data and the rotation angles of each joint of the slave terminal device.

[0042] The present invention has the following beneficial effects:

[0043] 1. The present invention implements the virtualization of the polishing process based on the Unity3D platform; the system not only provides a safe and efficient virtual training platform for operators, but also provides solid technical support for remote operation through precise master-slave device space mapping technology; by combining the Geomagic Touch force feedback device and the HTC VIVE Pro 2 virtual reality device, the embodiment realizes precise space mapping between heterogeneous models and constructs a realistic virtual polishing environment.

[0044] 2. In the present invention, precise robot arm control is achieved by designing a spatial mapping algorithm between heterogeneous models. The mesh model of the workpiece is generated by adding and allocating vertex positions and constructing triangular surfaces through the mesh subdivision method. The polishing function is realized based on the Mesh cutting algorithm. The force effect during the polishing process is provided by the force feedback device, thereby achieving the purpose of virtualizing the polishing process.

[0045] 3. In the present invention, the terminal posture of the main terminal device is mapped and sent to the virtual robotic arm, and the inverse kinematics of the robot is solved to calculate the joint angle for control, so as to achieve more precise and flexible control.

[0046] 4. The present invention provides users with real tactile and visual feedback by integrating the API of Geomagic Touch and HTC VIVE Pro 2, further enhancing the interactive experience.

[0047] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the invention, the drawings required for describing the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For ordinary technicians in this field, they can also obtain drawings based on these drawings without paying creative work.

[0049] Figure 1 Schematic diagram of the process of virtualization of the polishing process of the present invention, wherein (a) represents model preparation, (b) represents the master-slave space mapping scheme, and (c) represents the virtualization of the polishing process;

[0050] Figure 2 It is a schematic diagram of the DH coordinate system of Touch of the present invention;

[0051] Figure 3 Schematic diagram of the DH parameter coordinate system of the robot arm of the present invention;

[0052] Figure 4 This is a schematic diagram of the workspace mapping algorithm flow of the present invention;

[0053] Figure 5 This is a schematic diagram of the Touch workspace point cloud of the present invention;

[0054] Figure 6 Schematic diagram of the point cloud of the RM65-B workspace of the present invention; in the figure, X, Y, and Z respectively represent the positions of the robot end effector relative to the X-axis, Y-axis, and Z-axis of the base coordinate system; R, P, and Y respectively represent the posture of the robot end effector relative to the base coordinate system, that is, the rotation angles around the X, Y, and Z axes respectively; q1, q2, q3, q4, q5, and q6 respectively represent the rotation angles of the 1st, 2nd, 3rd, 4th, 5th, and 6th joints;

[0055] Figure 7 This is a schematic diagram of the comparison of the point clouds of the master and slave end workspaces after mapping in the present invention;

[0056] Figure 8 It is a schematic diagram of the projection of the point cloud comparison diagram of the master and slave end workspaces in the xy plane after mapping of the present invention;

[0057] Fig. 9 It is a schematic diagram of the projection of the point cloud comparison diagram of the master and slave end workspaces in the yz plane after mapping of the present invention;

[0058] Fig.10It is a schematic diagram of the projection of the point cloud comparison diagram of the master and slave end workspaces in the xz plane after mapping of the present invention;

[0059] Fig.11 It is a schematic diagram of a virtual scene of the present invention;

[0060] Fig.12 Schematic diagram of the correspondence between the left and right hand coordinate systems of the present invention, where θ L ,φ L , Respectively represent the rotation angles along the X, Y, and Z axes in the left-handed system; X L , Y L , Z L Respectively represent the X, Y, and Z axes in the left-handed system; X R , Y R , Z R Respectively represent the X, Y, and Z axes in the right-hand system; θ R ,φ R , They represent the rotation angles along the X, Y, and Z axes in a right-handed system respectively;

[0061] Fig.13 This is a schematic diagram of the Mesh cutting algorithm flow of the present invention;

[0062] Fig.14 This is a schematic diagram of the virtualization test of the polishing process of the present invention;

[0063] Fig.15 This is a schematic diagram of the virtualization test of the polishing process of the present invention;

[0064] Fig.16 This is a comparative diagram of the left-side posture mapping of the present invention;

[0065] Fig.17 This is a comparative diagram of the right-side mapping of the posture of the present invention;

[0066] Fig.18 The three figures of the present invention represent the effect diagram under the conditions of Table 4, wherein 1 represents the grinding effect diagram when the force is 1N and the grinding head moving speed is 8mm / s; 2 represents the grinding effect diagram when the force is 1N and the grinding head moving speed is 9mm / s; 3 represents the grinding effect diagram when the force is 1N and the grinding head moving speed is 10mm / s; 4 represents the grinding effect diagram when the force is 2N and the grinding head moving speed is 8mm / s; 5 represents the grinding effect diagram when the force is 2N and the grinding head moving speed is 9mm / s; 6 represents the grinding effect diagram when the force is 2N and the grinding head moving speed is 10mm / s; 7 represents the grinding effect diagram when the force is 3N and the grinding head moving speed is 8mm / s; 8 represents the grinding effect diagram when the force is 3N and the grinding head moving speed is 9mm / s; 9 represents the grinding effect diagram when the force is 3N and the grinding head moving speed is 10mm / s. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the invention to clearly and completely describe the technical solutions in the embodiments of the invention. Obviously, the described embodiments are only part of the embodiments of the invention, not all of the embodiments. Based on the embodiments in the invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the invention.

[0068] In the description of the present invention, it is necessary to understand that the terms "opening", "upper", "lower", "top", "middle", "inside" and the like indicating orientation or positional relationship are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the components or elements referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the invention.

[0069] Embodiment 1

[0070] Reference Figure 1-18 As shown, this embodiment is a polishing process virtualization simulation method based on Unity3D, comprising the following steps:

[0071] S1. Establish a model and convert it to construct a polishing robot model suitable for Unity3D, and obtain the final polishing robot model; set the master device and the slave device;

[0072] The master device in S1 includes Geomagic Touch, and the slave device includes RM65-B robotic arm;

[0073] The S1 comprises the following steps:

[0074] S11. Use SolidWorks software to accurately model the virtual grinding robot arm to obtain an initial robot arm model;

[0075] S12, using Blender software to convert the right-hand coordinate system of the initial mechanical arm model into the left-hand coordinate system required by Unity3D, and placing the initial mechanical arm model at the origin to eliminate the offset, thereby obtaining a converted mechanical arm model;

[0076] S13. According to the specific structure of the robotic arm, a mesh collider, a rigid body component and a material are added to the converted robotic arm model to obtain a final polished robotic arm model;

[0077] S2. Define the working space range of the master device and the slave device to obtain the working space range of the master end and the working space range of the slave end;

[0078] The S2 comprises the following steps:

[0079] S21, using the Monte Carlo method and combining the DH parameters of the master device and the slave device, as shown in Table 1 and Table 2; solving the working space of the master device and the slave device to obtain the working space diagram of the master device and the slave device;

[0080] Table 1

[0081]

[0082] Among them, a i-1 From Z i-1 to Z i The distance along X i-1 The axis is pointing positively; α i-1 From Z i-1 to Z i The corner along X i-1 The positive direction of the shaft rotation is positive; d i From X i-1 To X i The distance along the Z axis is positive; θ i From X i-1 To X i The rotation angle is positive along the positive direction of the Z axis; i-1 Represents the Z axis of the i-1th coordinate system, Z i represents the Z axis of the i-th coordinate system, X i-1 represents the X-axis of the i-1th coordinate system, X i represents the X-axis of the i-th coordinate system;

[0083] Combined with the DH parameters constructed in Table 1, the transformation matrix between the robot's adjacent coordinate systems i-1 and i is constructed as follows:

[0084]

[0085] in, Represents the transformation matrix between adjacent coordinate systems i-1 and i;

[0086] Then the posture transformation matrix of the robot end position relative to the base coordinate system is:

[0087]

[0088] Table 2

[0089]

[0090]

[0091] The rotation angle of each joint of the robot can be solved by inverse kinematics; the calculation formula is as follows:

[0092]

[0093] Represents the transformation matrix between coordinate system 0 and coordinate system 1, represents the transformation matrix between coordinate system 0 and coordinate system 6, represents the transformation matrix between coordinate system 1 and coordinate system 2, represents the transformation matrix between coordinate system 2 and coordinate system 3, represents the transformation matrix between coordinate system 3 and coordinate system 4, represents the transformation matrix between coordinate system 4 and coordinate system 5, represents the transformation matrix between coordinate system 5 and coordinate system 6;

[0094] With the help of MATLAB, a numerical method (Monte Carlo method) is used to solve the workspace of the master and slave devices by combining the DH parameters of the master and slave ends. In MATLAB, the Robotics Toolbox is used to set the DH parameters, establish the master and slave models, take the number of random values ​​N as 100,000, generate random values, solve the kinematic solution, and finally obtain the point cloud map of the master and slave workspaces.

[0095] S3, setting a mapping function and mapping function parameters according to the difference between the working space range of the master end and the working space range of the slave end; then obtaining each joint angle of the master end device in real time and using forward kinematics to calculate its posture to obtain posture data; then inputting the posture data into the mapping function to calculate the target posture of the slave end; and using inverse kinematics to calculate the rotation angle of each joint of the slave end device in conjunction with the target posture of the slave end;

[0096] The joints and structures of Geomagic Touch and RM65-B are different. In the design of heterogeneous robot systems, the difference in workspace between the master device and the slave robot is a key technical challenge. If the traditional joint space mapping method is used to directly map the joint angles of the master device to the slave, the working range will be limited and the control accuracy will be reduced. To overcome this problem, the end position of the master device is mapped and sent to the virtual robot, and the robot inverse kinematics is solved to calculate the joint angles for control, so as to achieve more precise and flexible control.

[0097] In S3, setting a mapping function according to the difference between the working space range of the master end and the working space range of the slave end includes the following steps:

[0098] S31, according to the master-end device and slave-end device workspace diagram, a table of the workspace range of the master-end device and the slave-end device is listed; as shown in Table 3,

[0099] Table 3

[0100]

[0101] S32, constructing a mapping function according to the working space range table of the master device and the slave device; as follows,

[0102] S = KL + B;

[0103] In the formula, S represents the end position vector of the master device, L represents the end target position vector of the robot, K represents the spatial mapping coefficient, and B represents the constant that the origin position of the master device workspace after proportional enlargement is consistent with the origin position of the robot workspace;

[0104] Assume S is the end position vector of Touch, L is the end target position vector of the robot arm (i.e. the mapped position vector), and the spatial mapping coefficient is K. Then there is the following relationship between S and L:

[0105] S = KL + B;

[0106] Where B is the origin position of the main end workspace after proportional enlargement, which is consistent with the origin position of the robot arm workspace. Then we can get

[0107]

[0108] Where S Max , S Min Indicates the maximum and minimum values ​​of the slave workspace; L Max , L Min Indicates the maximum and minimum values ​​of the master end working space; According to Table 3, the components of the proportional coefficient K and the constant B in the X, Y, and Z directions are as follows:

[0109]

[0110] In the formula, K x , K y , K z Respectively represent the components of the proportional coefficient K in the X, Y, and Z directions; B x , B y , B z They represent the components of the constant B in the X, Y, and Z directions respectively; S xMax , S xMin They represent the maximum and minimum values ​​of the slave end workspace in the X direction, S yMax , S yMin They represent the maximum and minimum values ​​of the slave end workspace in the Y direction, S zMax , S zMin They represent the maximum and minimum values ​​of the slave end working space in the Z direction respectively; L xMax , LxMin They represent the maximum and minimum values ​​of the master end workspace in the X direction, L yMax , L yMin They represent the maximum and minimum values ​​of the master workspace in the Y direction, L zMax , L zMin Respectively represent the maximum and minimum values ​​of the master end workspace in the Z direction;

[0111] According to Table 3, K x , K y , K z They are 4.96, 3.25, 3.47 respectively, B x , B y , B z They are -548.25mm, 18.96mm, and -567.96mm respectively;

[0112] S4, realize the virtualization of the polishing process by cooperating with the final polishing robot arm model, posture data and the rotation angles of each joint of the slave device;

[0113] The S4 comprises the following steps:

[0114] S41, performing heterogeneous control, polishing function development and VR interaction to build a virtual scene in cooperation with the final polishing robot arm model, posture data and the rotation angles of each joint of the slave device, to obtain a built virtual scene;

[0115] The heterogeneous control in S41 includes the following steps:

[0116] S4111. Position and attitude are usually used to determine the state of the end of the robot in three-dimensional space;

[0117] S4112, converting the left-hand coordinate system into the right-hand coordinate system according to the state of the end of the robotic arm in the three-dimensional space;

[0118] The S4112 includes the following steps:

[0119] S41121, let the displacement along the X, Y, and Z axes in the left-hand system be P XL , P YL , P ZL , the rotation angle is θ L ,φ L , The corresponding displacement of the right-hand system is P XR , P YR , P ZR , the rotation angles are θ R ,φ R , From this, we can see that there is the following mathematical mapping relationship between the left-handed coordinate system posture and the right-handed coordinate system posture;

[0120]

[0121] S4113, real-time acquisition of the joint angle data of the master-end device, and the preparation of a forward kinematics script to calculate the terminal posture in conjunction with the final polishing robot arm model, posture data, and the rotation angles of each joint of the slave-end device, and the posture is converted into posture data that can be recognized by the robot arm, and finally the heterogeneous model control script is used to realize precise control of the master and slave ends;

[0122] The polishing function development described in S41 includes the following steps:

[0123] S4121. First, set the attributes of the model and the number of mesh subdivisions. Use the mesh subdivision method to add vertices to create the model mesh, calculate the position of each vertex in the scene, set the vertex coordinate position array of each face of the workpiece based on the local coordinates of the vertex, and then set the index array (Triangles) of the triangles of each face of the workpiece according to the vertex subscript, dynamically create a mesh model of the workpiece, and add material to it.

[0124] S4122, create a grinding head, convert the world coordinates of the grinding head into local coordinates, loop through all vertex arrays of the workpiece vertices, first determine whether the y value of the vertex coordinate of the grinding head is less than the y value of the vertex coordinate of the workpiece, then screen whether the vertex coordinate of the workpiece is within the radius range of the grinding head in the two-dimensional plane, if it is within the radius range, continue to screen all vertices within the radius range in the three-dimensional plane, if it is within this range, dynamically modify the y value of the vertex coordinate of the workpiece to the y value of the local coordinate of the grinding head, thereby simulating material removal, and finally update the vertex information, recalculate the normal and tangent of the workpiece mesh, and update the Mesh rendering to realize the grinding function;

[0125] S42, performing a grinding operation simulation operation in the constructed virtual scene to realize grinding process virtualization;

[0126] In the virtual simulation system, model building and tactile feedback are two core elements to achieve realistic operation experience. In order to ensure the accuracy and functionality of the model in Unity3D, the virtual grinding robot arm is firstly accurately modeled by SolidWorks software. Secondly, in order to solve the difference in coordinate systems between different software, the right-hand coordinate system of the model is converted to the left-hand coordinate system required by Unity3D by using Blender software, and the model is placed at the origin to eliminate the offset. Thirdly, the robot components are organized by establishing a parent-child relationship, so that the child object inherits the parent object transformation and forms an orderly hierarchical structure, which is convenient for complex movement and structural organization. Finally, in order to achieve accurate collision detection and tactile feedback, mesh colliders, rigid body components and materials are added to the robot arm according to its specific structure. At the same time, in order to simulate the real tactile interaction situation, the workpiece also needs to add rigid body, collider and material. The constructed virtual scene includes elements such as virtual grinding robot arm, HTC VIVE Pro2 camera, Touch, workpiece, etc., creating a highly realistic and interactive virtual environment.

[0127] When implementing heterogeneous model control in a virtual scene, coordinate system conversion plays a vital role. The state of the end of the robot in three-dimensional space is usually expressed in position and posture. The position and posture obtained by the Transform component of Unity3D are in the left-handed coordinate system, but the kinematic modeling of the robot arm usually uses the right-handed coordinate system. In order to ensure the control accuracy of the master and slave ends, the left-handed coordinate system must be converted to the right-handed coordinate system.

[0128] To achieve precise control of heterogeneous models, first obtain the Touch joint angle data in real time, then write a forward kinematics script to calculate the end pose, and then use the above coordinate system conversion to convert the pose data that can be recognized by the robot arm. Finally, use the heterogeneous model control script to achieve precise control of the master and slave ends.

[0129] In order to realize the virtualization of the process and the integration of visual information, this embodiment adopts an improved Mesh cutting algorithm in Unity3D to dynamically modify the mesh vertices of the model at runtime to simulate the removal of materials during the grinding process. The specific steps are as follows:

[0130] (1) Create a mesh model for the workpiece: First, set the model's attributes and the number of mesh subdivisions. Use the mesh subdivision method to add vertices to create the model mesh and calculate the position of each vertex in the scene. Set the vertex coordinate position array (Vertices) of each face of the workpiece based on the local coordinates of the vertex. Then, set the index array (Triangles) of the triangles that make up each face of the workpiece according to the vertex subscript. Dynamically create a mesh model for the workpiece and add materials to it.

[0131] (2) Execute the Mesh cutting algorithm: create a grinding head, convert the world coordinates of the grinding head into local coordinates, loop through all vertex arrays of the workpiece vertices, first determine whether the y value of the vertex coordinate of the grinding head is less than the y value of the vertex coordinate of the workpiece, then screen the workpiece vertex coordinate in the two-dimensional plane to see if it is within the radius range of the grinding head. If it is within the radius range, continue to screen all vertices within the radius range in the three-dimensional plane. If it is within this range, dynamically modify the y value of the vertex coordinate of the workpiece to the y value of the local coordinate of the grinding head, thereby simulating the removal of material. Finally, update the vertex information, recalculate the normal and tangent of the workpiece mesh, and update the Mesh rendering to realize the grinding function.

[0132] Finally, virtual testing of the polishing process is carried out;

[0133] Experimental conditions:

[0134] This test was conducted under specific system environment and software version conditions, namely 64-bit Win11 system, Unity3D 2021.3.32f1c1 version and Visual Studio 2022 version; the polishing process virtualization test scenario is as follows Fig.10 As shown in the figure, it covers the main device Geomagic Touch, HTC VIVE Pro 2 head-mounted display, locator, computer and stepper motor controller kit (the kit includes stepper motor driver, KH-01 stepper motor controller, 24V DC switching power supply and ball screw linear slide module); Touch and slide rail are connected by using 3D printed clamping device, in which the ball screw mechanism plays a key role. It can convert the rotational motion of the motor into translational motion, ensuring the steady speed and smooth trajectory during the grinding process, thereby driving the linear motion of the main device to realize the grinding operation; when the user wears HTC VIVE Pro 2, they can get a more realistic experience and simulate the grinding operation process more accurately;

[0135] Experiment and result analysis:

[0136] In order to verify the correctness of the operation of the virtualization of the polishing process proposed in this embodiment and the accuracy of the master-slave mapping, the experiment was conducted by having the operator wear HTC VIVE Pro 2 to operate the master device to control the movement of the virtual robotic arm. The master camera was used to observe the movement of the virtual robotic arm in real time, and the coordinate position of the camera was adjusted so that the virtual model could fully appear in the camera screen during movement for easy observation.

[0137] In order to verify the accuracy of the material removal depth during the grinding process, this embodiment uses a brown corundum spherical grinding head, and the material of the workpiece is titanium alloy (TC4), whose density is 4.43g / m 3, the expression of material removal depth is as follows,

[0138] d=K c V MRR / W B v ω ;

[0139] Where d is the material removal depth, unit: mm, K c is the material removal factor related to the material and grinding tool characteristics, V MRR is the material removal rate, in mm 3 / s,W B is the width of the abrasive belt, in mm, v ω is the feed speed of the workpiece, in mm / s;

[0140] According to the principle of energy conservation, the work done by the spherical grinding head during the grinding process is equal to the energy of material removal. The work done by the grinding head can be calculated by the product of force and displacement, and the energy of material removal is related to the density and volume of the material. The material removal rate of the spherical grinding head is expressed as follows:

[0141] V MRR =V / t=Fv ω t / ρtπr 2 =Fv ω / ρπr 2 ;

[0142] Where V is the material removal amount, unit: mm 3 , v is the linear velocity of the spherical grinding head, in mm / s, F is the force applied during the grinding process, in N, in mm / s, t is the contact time between the grinding head and the workpiece, in s, ρ is the density of the workpiece, in g / mm 3 , r is the radius of the spherical grinding head; the material removal depth of the spherical grinding head is obtained as:

[0143] d=50K c F / ρr 2 v h ;

[0144] K c is the material removal coefficient related to the characteristics of the material and the grinding tool, with a value of 0.065, v h is the moving speed of the grinding head; is the relationship between different forces, grinding head moving speed and material removal depth in the embodiment, the number of grinding times is set to 20 times, and the grinding parameters are shown in Table 4,

[0145] Table 4

[0146] ;

[0148] In the experiment of this embodiment, the force in the grinding process is controlled by controlling the height of the clamped Touch, that is, controlling the height between the Touch and the workpiece in the scene. The device will display the force in the scene, which is set to 1N, 2N, and 3N. The speed program is written through the KH-01 stepper motor controller to achieve 8mm / s, 9mm / s, and 10mm / s grinding head speeds. The simulation experiment is carried out according to the parameters in Table 4. The simulation results are as follows: Fig.11 As shown in the figure, the serial numbers correspond to the serial numbers in the table; the experimental results show that the system can accurately simulate the material removal effect during the grinding process. By changing the grinding force and speed, the change in material removal depth can be clearly observed. Specifically, when the grinding head speed remains unchanged, increasing the force will lead to an increase in material removal depth; when the grinding force remains unchanged, reducing the grinding head speed will increase the material removal depth;

[0149] Summarize:

[0150] To provide users with a safe and economical experimental environment, this embodiment implements the virtualization of the grinding process based on the Unity3D platform; the system not only provides operators with a safe and efficient virtual training platform, but also provides solid technical support for remote operation through precise master-slave device space mapping technology; by combining the Geomagic Touch force feedback device and the HTC VIVE Pro 2 virtual reality device, the embodiment realizes precise space mapping between heterogeneous models and constructs a realistic virtual grinding environment; the mesh subdivision method and the improved Mesh cutting algorithm are used to dynamically construct the mesh model of the workpiece, and the material removal effect during the grinding process is simulated, which enhances the realism of training and remote operation; in addition, by integrating the API of Geomagic Touch and the HTC VIVE Pro 2, real tactile and visual feedback is provided to users, further enhancing the interactive experience; experimental verification shows that the system can effectively simulate the grinding operation, and reveals the relationship between force, grinding head speed and material removal depth, proving the effectiveness and application potential of the virtual simulation system in education and training and remote operation control; future work will be extended to consider the change of workpiece surface roughness during the grinding process to further enhance the authenticity and effectiveness of the virtual training system and remote operation technology;

[0151] Figure 1 The process is described as follows:

[0152] Model preparation: Use SolidWorks software to accurately model the virtual grinding robot arm, save it as an STL file, and then use Blender software to convert the coordinate system, that is, convert the right-hand coordinate system of the model in the STL file to the left-hand coordinate system required by Unity3D, and then set the rotation axis and scale size for the model in the converted STL file, and export the corresponding FBX file to obtain the virtual RM65-B model; Virtualization of grinding process: Build a virtual scene. First, convert the joint angle of Geomagic Touch into the joint angle of the virtual robot arm through the master-slave space mapping script, and transmit it to the robot arm, so as to achieve precise control of the RM65-B model. Second, develop the grinding function and dynamically create a network model of the workpiece; third, use VR interaction technology, that is, use the SteamVR plug-in and HTC VIVE Pro 2, and cooperate with the improved Mesh cutting algorithm, the network model of the workpiece and the RM65-B model to simulate the grinding operation;

[0153] Among them, STL file is a file format used to represent triangular meshes in computer graphics application systems; Blender is an open source 3D creation suite that supports the entire 3D creation process, including modeling, sculpting, skeleton assembly, animation, simulation, real-time rendering, synthesis and motion tracking; FBX file is a 3D data exchange file format developed by Autodesk; Geomagic Touch is a professional force feedback device; RM65-B is an ultra-lightweight humanoid robotic arm produced by Realman; VR interaction refers to the interaction and communication between users and virtual reality environments; SteamVR plug-in is a set of software tools developed by Valve Corporation; HTC VIVE Pro 2 is a high-end virtual reality head-mounted display;

[0154] Fig.13 The process is described as follows:

[0155] At the beginning of the process, initialize the Mesh, set the Mesh properties, set the number of Mesh subdivisions, add vertices and triangular faces to create the model mesh through the mesh subdivision method, and calculate the position of each vertex in the scene. Set the vertex coordinate position array of each face of the workpiece based on the local coordinates of the vertex, and then set the index array of the triangular faces of each face of the workpiece according to the vertex subscript, dynamically create the Mesh model of the workpiece, and add materials to it; determine whether the Mesh needs to be updated. If not, the process ends; if it needs to be updated, execute the Mesh cutting algorithm: create a grinding head, convert the world coordinates of the grinding head to local coordinates, and loop through all vertex arrays of the workpiece vertices First, determine whether the y value of the vertex coordinate of the grinding head is less than the y value of the vertex coordinate of the workpiece. When the y value of the vertex coordinate of the grinding head is not less than the y value of the vertex coordinate of the workpiece, the process ends; otherwise, determine whether the vertex is in the two-dimensional plane of the radius. When the vertex is not in the two-dimensional plane of the radius, the process ends; otherwise, continue to screen all vertices within the radius in the three-dimensional plane, and determine whether there are vertices in the three-dimensional plane within the radius. If not, the process ends; otherwise, dynamically modify the y value of the vertex coordinate of the workpiece to the y value of the local coordinate of the grinding head, and finally update the vertex information, recalculate the normal and tangent of the workpiece mesh, and update the Mesh rendering to realize the grinding function, and the process ends.

[0156] Embodiment 2

[0157] This embodiment discloses a polishing process virtualization simulation system based on Unity3D, which can implement the method of the above embodiment, including a model construction conversion module, a workspace scope definition module, a master-slave end mapping construction module and a polishing process virtualization module;

[0158] The model building conversion module is used to build and convert the model, build a polishing robot model suitable for Unity3D, obtain the final polishing robot model and set the master end device and the slave end device;

[0159] The working space range defining module is used to define the working space ranges of the master end device and the slave end device, and obtain the master end working space range and the slave end working space range;

[0160] The master-slave mapping construction module is used to set a mapping function and mapping function parameters according to the difference between the master workspace range and the slave workspace range; then obtain each joint angle of the master device in real time and use forward kinematics to calculate its posture to obtain posture data; then input the posture data into the mapping function to calculate the target posture of the slave; and use inverse kinematics to calculate the rotation angle of each joint of the slave device in conjunction with the target posture of the slave;

[0161] The polishing process virtualization module is used to realize the virtualization of the polishing process in conjunction with the final polishing robot arm model, posture data and the rotation angles of each joint of the slave terminal device.

[0162] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0163] The preferred embodiments of the invention disclosed above are only used to help explain the invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the invention, so that those skilled in the art can understand and use the invention well.

Claims

1. A polishing process virtualization simulation method based on Unity3D, characterized in that: The following steps are involved: S1. Establish a model and convert it to construct a polishing robot model suitable for Unity3D, and obtain the final polishing robot model; set the master device and the slave device; S2. Define the working space range of the master device and the slave device to obtain the working space range of the master end and the working space range of the slave end; S3, setting a mapping function and mapping function parameters according to the difference between the working space range of the master end and the working space range of the slave end; then obtaining each joint angle of the master end device in real time and using forward kinematics to calculate its posture to obtain posture data; then inputting the posture data into the mapping function to calculate the target posture of the slave end; Cooperate with the target posture of the slave end and use inverse kinematics to calculate the rotation angle of each joint of the slave end device; S4. Virtualize the polishing process by coordinating the final polishing robot arm model, posture data, and the rotation angles of each joint of the slave device.

2. The Unity3D-based polishing process virtualization simulation method according to claim 1, characterized in that: The master device in S1 includes Geomagic Touch, and the slave device includes RM65-B robotic arm.

3. The Unity3D-based polishing process virtualization simulation method according to claim 1, characterized in that: The S1 comprises the following steps: S11. Use SolidWorks software to accurately model the virtual grinding robot arm to obtain an initial robot arm model; S12, using Blender software to convert the right-hand coordinate system of the initial mechanical arm model into the left-hand coordinate system required by Unity3D, and placing the initial mechanical arm model at the origin to eliminate the offset, thereby obtaining a converted mechanical arm model; S13. According to the specific structure of the robotic arm, a mesh collider, a rigid body component and a material are added to the converted robotic arm model to obtain a final polished robotic arm model.

4. The Unity3D-based polishing process virtualization simulation method according to claim 3, characterized in that: The S2 comprises the following steps: S21. Using the Monte Carlo method and combining the DH parameters of the master-end device and the slave-end device, the working spaces of the master-end device and the slave-end device are solved to obtain working space diagrams of the master-end device and the slave-end device.

5. The Unity3D-based polishing process virtualization simulation method according to claim 4, characterized in that: In S3, setting a mapping function according to the difference between the working space range of the master end and the working space range of the slave end includes the following steps: S31, listing a table of working space ranges of the master-end device and the slave-end device according to the working space diagrams of the master-end device and the slave-end device; S32, constructing a mapping function according to the working space range table of the master device and the slave device; as follows, S = KL + B; Where S represents the end position vector of the master device, L represents the end target position vector of the robot, K represents the spatial mapping coefficient, and B represents the constant that keeps the origin position of the master device workspace after proportional enlargement consistent with the origin position of the robot workspace.

6. The Unity3D-based polishing process virtualization simulation method according to claim 5, characterized in that: The S4 comprises the following steps: S41, performing heterogeneous control, polishing function development and VR interaction to build a virtual scene in cooperation with the final polishing robot arm model, posture data and the rotation angles of each joint of the slave device, to obtain a built virtual scene; S42, performing a grinding operation simulation operation in the constructed virtual scene to realize the virtualization of the grinding process.

7. The Unity3D-based polishing process virtualization simulation method according to claim 6, characterized in that: The heterogeneous control in S41 includes the following steps: S4111. Position and attitude are usually used to determine the state of the end of the robot in three-dimensional space; S4112, converting the left-hand coordinate system into the right-hand coordinate system according to the state of the end of the robotic arm in the three-dimensional space; S4113. Acquire the joint angle data of the master-end device in real time, and write a forward kinematics script to calculate the terminal posture in conjunction with the final polishing robot arm model, posture data, and the rotation angles of each joint of the slave-end device, convert the posture into posture data that can be recognized by the robot arm, and finally use the heterogeneous model control script to achieve precise control of the master and slave ends.

8. The Unity3D-based polishing process virtualization simulation method according to claim 7, characterized in that: The S4112 includes the following steps: S41121, let the displacement along the X, Y, and Z axes in the left-hand system be P XL , P YL , P ZL , the rotation angle is θ L ,φ L , The corresponding displacement of the right-hand system is P XR , P YR , P ZR , the rotation angles are θ R ,φ R , Construct the mapping relationship between the left-handed coordinate system pose and the right-handed coordinate system pose.

9. The Unity3D-based polishing process virtualization simulation method according to claim 8, characterized in that: The polishing function development described in S41 includes the following steps: S4121, firstly, set the attributes of the model and the number of mesh subdivisions, add vertices of the model mesh through the mesh subdivision method, calculate the position of each vertex in the scene, set the vertex coordinate position array of each face of the workpiece based on the local coordinates of the vertex, then set the index array of the triangle surface of each face of the workpiece according to the vertex subscript, dynamically create the mesh model of the workpiece, and add material to it; S4122. Create a grinding head, convert the world coordinates of the grinding head into local coordinates, loop through all vertex arrays of the workpiece vertices, first determine whether the y value of the vertex coordinate of the grinding head is less than the y value of the vertex coordinate of the workpiece, then screen the vertex coordinate of the workpiece in the two-dimensional plane to see if it is within the radius range of the grinding head. If it is within the radius range, continue to screen all vertices within the radius range in the three-dimensional plane. If it is within this range, dynamically modify the y value of the vertex coordinate of the workpiece to the y value of the local coordinate of the grinding head, thereby simulating material removal. Finally, update the vertex information, recalculate the normal and tangent of the workpiece mesh, and update the Mesh rendering to realize the grinding function.

10. A system for implementing a Unity3D-based polishing process virtualization simulation method as described in any one of claims 1 to 9.

Images