Simulation method and system of spraying mechanical arm based on Bullet engine

Abstract

The invention discloses a simulation method and system for a spraying mechanical arm based on a Bullet engine. The simulation method comprises the following steps that SolidWorks software is combined with a URDF plug-in, three-dimensional modeling is conducted based on spraying mechanical arm model data, a basic axis and a coordinate system are established at the joint of the spraying mechanical arm, kinematics parameters are set, and a URDF file of the spraying mechanical arm is exported; on the basis of a Bullet engine, motion control over the spraying mechanical arm is achieved in a simulation environment in combination with a forward kinematics equation or a reverse kinematics equation; on the basis of a Bullet engine, multiple working scenes of the spraying mechanical arm are built and added into a model of the spraying mechanical arm; and scanning the steel structural part and forming point cloud data, generating a spraying track of the spraying mechanical arm by combining the spraying process parameters of the steel structural part, and importing the spraying track into the spraying mechanical arm model in the simulation environment for verification.

Description

Technical Field

[0001] The present invention relates to the field of spraying control and simulation of a robot arm, and in particular to a simulation method and system of a spraying robot arm based on a Bullet engine. Background Art

[0002] In the field of industrial manufacturing, the use of robotic arms is becoming more and more widespread, and more and more manufacturing tasks use robotic arms to replace manual labor to complete various production tasks. However, due to the high cost of robotic arms and the complexity of the production tasks, if actual experimental operations are carried out directly, unnecessary risks and extremely high cost losses will occur. At the same time, due to the complexity of the real environment and the emergence of force majeure, professionals are required to make correct judgments to solve various emergencies. This poses a great challenge to industrial production.

[0003] In industrial production, spraying of steel components is an indispensable part of the production process of steel components. It is not only a means to beautify the appearance of steel components, but also an important part to protect steel components from corrosion and extend the life of steel components. However, most parts of my country still rely on traditional manual methods for spraying in the manufacturing industry. In order to meet the user's requirements for paint film thickness and surface quality, additional grinding and re-spraying are often required. As a result, the labor intensity of manual spraying is high, the operation efficiency is low, and long-term contact with paint and solvents is harmful to the workers' bodies. Therefore, using machines to replace people to improve production efficiency, improve quality and reduce operating costs is the choice of more companies.

[0004] Since the cost of the robot arm is high, if the steel component spraying experiment is carried out directly, the robot arm may collide with the steel component, causing damage to the robot arm. At the same time, the spraying experiment will waste the spraying material, which will lead to an increase in production costs. Therefore, it is necessary to build a robot arm spraying simulation platform to allow staff to simulate and verify the spraying behavior in a virtual environment to guide subsequent spraying operations and reduce unnecessary losses and waste. However, the existing methods of spraying steel components by robot arms are mostly designed based on a certain type of steel component. If a new shape of steel component is replaced, the staff needs to redesign a set of spraying trajectories to implement the new steel component, resulting in a decrease in production efficiency. At the same time, the simulation platform is relatively cumbersome to design a set of robot motion processes, and often requires users to have strong programming skills and system configuration knowledge to define complex motion trajectories and operation sequences. This means that non-professional users may need to spend a lot of time to learn and adapt to the use of the platform, which increases the initial learning cost. Secondly, when these platforms deal with specific fields such as spraying processes, they may lack preset professional modules and algorithms, causing users to have to build from the bottom up. Summary of the invention

[0005] In response to the problems and needs raised above, this solution proposes a simulation method and system for a spray robot arm based on the Bullet engine. Due to the adoption of the following technical features, the above technical objectives can be achieved and many other technical effects can be brought about.

[0006] An object of the present invention is to provide a simulation method for a spraying robot arm based on a Bullet engine, comprising the following steps:

[0007] S10: Use SolidWorks software combined with URDF plug-in to perform 3D modeling based on the spraying robot arm model data, establish basic axes and coordinate systems at the joints of the spraying robot arm, set corresponding kinematic parameters, and export the URDF file of the spraying robot arm;

[0008] S20: Based on the Bullet engine, the forward kinematics equations of the motion angles of each joint of the spraying robot arm are calculated according to the coordinates and posture of the end of the spraying robot arm, or the end posture of the spraying robot arm is calculated through the motion angles of each joint of the spraying robot arm, and the inverse kinematics equations of controlling the spraying robot arm to move to the specified position are used to realize the motion control of the spraying robot arm in the simulation environment;

[0009] S30: Based on the Bullet engine, build various working scenes of the spraying robot arm and create them in STL file format, select the corresponding working environment type and add the STL format file of the working environment type to the model of the spraying robot arm;

[0010] S40: Scan the steel structure and form point cloud data, and generate the spraying trajectory of the spraying robot arm in combination with the spraying process parameters of the steel structure, and import the spraying trajectory into the spraying robot arm model in the simulation environment for verification. If no error occurs during the simulation verification process, save the spraying trajectory data; if an error occurs during the simulation process, perform corresponding correction and optimization processing according to the error type until the simulation is correct.

[0011] In addition, the simulation method and system of the spraying robot arm based on the Bullet engine according to the present invention may also have the following technical features:

[0012] In an example of the present invention, step S20 includes the following steps:

[0013] S21: Construct the kinematic model of the robot arm in the simulation system. The spray robot arm consists of serially connected joints and links. For a robot arm with n degrees of freedom, it has n joints, the joints are numbered from 1 to n, there are n+1 links, the links are numbered from 0 to n, link 0 is the base of the robot arm, link n has an end effector, and joint i connects link i and link i-1; Based on the DH parameters, define the transformation matrix T of each jointi :

[0014]

[0015] Among them, a i It is along X i Axis, from Z i Move to Z i+1 The distance, i.e., the abstract length of connecting rod i; θ i It is around Z i Axis, from X i-1 Rotate to X i The angle between connecting rod i-1 and connecting rod i; α i From Z i Rotate to Z i+1 Angle, the axis of rotation is X i , that is, the angle between connecting rod i and connecting rod i+1; d i It is along Z i Axis, from X i-1 Move to X i distance;

[0016] S22: Solve the forward kinematics equation to calculate the end pose or solve the inverse kinematics equation to calculate the joint angle;

[0017] Solving the forward kinematics equations to compute the end pose involves:

[0018] The end pose is calculated by multiplying the transformation matrices of all joints:

[0019] T(q)=T1·T2……T n

[0020] The end position and posture are separated into:

[0021]

[0022] Among them, q is the motion angle of the joint, T(q) is the solution of positive kinematics, that is, the end position, p(q) is the end coordinate, and p(q) is the end posture.

[0023] Solving the inverse kinematics equations to calculate joint angles includes:

[0024] Calculate the current end pose error:

[0025] Position error:

[0026] Δp=p d -p(q)

[0027] Attitude error:

[0028] ΔR=R d ·R(q) T

[0029] By converting ΔR into a rotation vector Δr, we get:

[0030] Δr=logΔR

[0031] Comprehensive error:

[0032] ΔT=[Δp,Δr]

[0033] Compute the Jacobian matrix:

[0034] The Jacobian matrix J(q) represents the partial derivatives of the end pose with respect to the joint angle:

[0035]

[0036] Joint variable update:

[0037] Update the joint angles by solving linear equations:

[0038] Δq=J(q) -1 ΔT

[0039] Update joint angles:

[0040] q new =q+Δq

[0041] Check convergence:

[0042] If the error ||ΔT||<ε, the iteration is exited, and the joint angle solved by inverse kinematics is q new ;

[0043] S23: When solving the inverse kinematics equation to calculate the joint angle, the joint sequence number set in the URDF file is used to control the movement of each joint of the spray robot arm to the joint angle solved by the inverse kinematics, thereby realizing the motion control of the robot arm in the simulation system.

[0044] In one example of the present invention, in step S22, in the process of solving the joint motion angle by the inverse kinematics algorithm according to the pose information of the target point, it also includes: combining the interpolation algorithm to generate a series of discrete intermediate points between the starting point and the target point, so that the end effector of the robot arm moves along a straight line, and the specific steps are as follows:

[0045] S221: First, interpolate the coordinates of the two points. For linear motion, use the parameter λ∈[0,1] to represent the ratio from the starting point to the end point:

[0046] p(λ)=(1-λ)·p s +λ·p e

[0047] Among them, ps is the coordinate information of the starting point, p e is the coordinate information of the end point and the coordinate information of the interpolation point p;

[0048] According to the number of discretized points N from the starting point to the end point, calculate λ i =i / N, generate position sequence:

[0049] p i = p(λ i )

[0050] S222: Use the quaternion interpolation algorithm for the two-point posture, and convert the rotation matrix R of the starting point and the end point s and R e Convert to quaternion q s and q e , use the spherical linear interpolation algorithm to calculate the quaternion q of the interpolation point i , the quaternion q of the interpolation point i Then convert back to the terminal posture R i ; Among them, the quaternion q i The expression is as follows:

[0051]

[0052] Where θ = cos -1 (q s ·q e ) is the angle of the quaternion;

[0053] S223: For each interpolation point coordinate p i and the terminal posture R i , combined to generate the interpolation point pose P i =[p i ,R i ], and save the result in the interpolation point sequence P;

[0054] S224: Calculate the joint angle of the robot arm for each interpolation point through the inverse kinematics algorithm, and gradually send joint angle instructions according to the interpolation results to control the movement of the robot arm to achieve linear movement of the robot arm in the simulation system.

[0055] In an example of the present invention, step S40 specifically includes the following steps:

[0056] S41: acquiring point cloud data of each angle of the steel structure by a multi-angle scanner, and obtaining the overall point cloud data of the steel structure by point cloud registration;

[0057] S42: dividing the obtained overall point cloud data of the steel structure into cross-section data and vertical plate data, and pre-processing the cross-section data and the vertical plate data;

[0058] S43: Calculate the spraying range of the spraying robot arm in combination with the process parameters of the steel structure, and calculate the number of spraying tracks according to the spraying range of the spraying robot arm;

[0059] S44: Calculate characteristic points of the steel structure according to the number of spraying trajectories and the spraying range, and convert the characteristic points of the steel structure into the spatial position and spraying posture of the robot arm corresponding to each trajectory;

[0060] S45: Import the final spraying trajectory of the spraying robot arm into the spraying robot arm model in the simulation environment for verification.

[0061] In an example of the present invention, in step S41, obtaining the overall point cloud data of the steel structure through point cloud registration includes the following steps:

[0062] S411: Performing rough registration, firstly performing fast point feature histogram feature extraction on the source point cloud and the target point cloud respectively, obtaining the FPFH feature vector of each point, then using the feature matching algorithm to find similar feature vectors in the source point cloud and the target point cloud, and calculating the distance between the two FPFH feature vectors, wherein if the distance is small enough, they are considered to be corresponding point pairs; after finding a set of initial feature matching pairs, using the RANSAC algorithm to estimate the initial transformation matrix, wherein the RANSAC algorithm finds the optimal transformation matrix through random sampling and iterative optimization, and finally obtains the rotation matrix R and translation vector T of rough registration;

[0063] S412: Perform fine registration, use the initial transformation matrix obtained by coarse registration, and use the ICP algorithm to iteratively optimize to obtain a more accurate solution. The ICP algorithm calculates the distance between the corresponding points of the source point cloud and the target point cloud, constructs the rotation and translation matrix RT, transforms the source point cloud through RT, and calculates the mean square error after the transformation; if the mean square error meets the threshold condition, the algorithm ends, otherwise it continues to iterate until the error meets the threshold condition or the number of iterations is terminated, and finally the overall point cloud data of the steel structure is obtained.

[0064] In one example of the present invention, in step S42, dividing the obtained overall point cloud data of the steel structure into cross-section data and vertical plate data specifically includes the following steps:

[0065] S421: plane cutting the overall point cloud data of the steel structure along its length extension direction in units of K millimeters to obtain multiple three-dimensional cutting sections;

[0066] S422: According to the type of steel structure, it is divided into two categories: H-shaped steel and square steel. For the cross-section processing of H-shaped steel, the web is used as the symmetry center to divide the cross-section into two independent characteristic structures on the left and right; for the cross-section of square steel, each side of the square cross-section is used as the dividing plane to divide the cross-section into four independent characteristic structures;

[0067] S423: The characteristic structure divided by the cutting plane is processed as a basic unit, and its envelope data is extracted and described. Finally, the point cloud data of the entire steel structure is further subdivided into cross-section data and facade data.

[0068] In an example of the present invention, in step S43, the spraying range of the spraying robot arm is calculated in combination with the process parameters of the steel structure, wherein the spraying range calculation formula is as follows:

[0069]

[0070] Among them, S is the spraying range, D is the height of the nozzle from the board surface, and A is the nozzle angle.

[0071] In one example of the present invention, in step S43, calculating the number of spraying tracks according to the spraying range of the spraying robot arm includes the following steps:

[0072] S431: Based on the spraying margin in the process parameters and the overlap of the two spraying trajectories, start calculating the first starting point of each set of data; wherein the spraying margin is defined as the increment or decrement of the amount of spraying of the next set after the robot arm completes each set of spraying tasks; the overlap of the two spraying trajectories is deducted from the overall spraying range, and the actual spacing between the trajectories is calculated to further determine the spraying path of the next trajectory;

[0073] S432: After determining the coordinates of the first point, the remaining spraying range is calculated through the first point and the range of each set of point cloud data;

[0074] S433: Divide the remaining spraying range by the distance between the two tracks to calculate how many tracks are needed to spray the area;

[0075] S434: Multiply the number of trajectories by the distance between the trajectories and subtract the remaining spraying range to obtain the remainder of the current set of spraying data, which will be used as a reference for calculating the starting point of the next set of data.

[0076] In one example of the present invention, in step S44, the characteristic points of the steel structure are calculated according to the number of spraying trajectories and the spraying range, and the characteristic points of the steel structure are converted into the spatial position of the robot arm and the spraying posture corresponding to each trajectory, which specifically includes the following steps:

[0077] S441: The first starting point position plus N multiplied by the distance between the two spraying tracks is used to calculate the feature point position of the Nth track;

[0078] S442: According to the characteristic points of the Nth trajectory, the cross-section data and the vertical plate data within the spraying range of each trajectory are traversed, and the starting point, the ending point, the highest point and the lowest point of each spraying trajectory are determined according to the four key points obtained after preprocessing;

[0079] S443: After obtaining the trajectory spraying coordinates, they need to be converted into the spatial position (x, y, z) of the robot arm and the corresponding spraying posture (Rx, Ry, Rz);

[0080] S444: Save the spraying robot arm posture data of each spraying trajectory into a list to form complete spraying trajectory coordinate data.

[0081] Another object of the present invention is to provide a simulation system for a spraying robot arm based on a Bullet engine, comprising:

[0082] A robot arm model creation module is configured to use SolidWorks software in combination with a URDF plug-in to perform three-dimensional modeling based on the spraying robot arm model data, establish basic axes and coordinate systems at the joints of the spraying robot arm, set corresponding kinematic parameters, and export a URDF file of the spraying robot arm;

[0083] The robot control module is configured to calculate the forward kinematics equations of the motion angles of each joint of the spraying robot arm based on the Bullet engine, or calculate the end position of the spraying robot arm through the motion angles of each joint of the spraying robot arm, and control the spraying robot arm to move to a specified position by using the inverse kinematics equations, so as to realize the motion control of the spraying robot arm in the simulation environment;

[0084] The work scene import module is configured to build various work scenes of the spraying robot arm based on the Bullet engine and create them in the STL file format, select the corresponding work environment type and add the STL format file of the work environment type to the model of the spraying robot arm;

[0085] The spraying trajectory calculation module is configured to scan the steel structure and form point cloud data, and generate the spraying trajectory of the spraying robot arm in combination with the spraying process parameters of the steel structure, and import the spraying trajectory into the spraying robot arm model in the simulation environment for verification. If no error occurs during the simulation verification process, the spraying trajectory data is saved; if an error occurs during the simulation process, corresponding correction and optimization processing is performed according to the error type until the simulation is correct.

[0086] Compared with the prior art, the present invention has the following beneficial effects:

[0087] 1. Adapting to various robot arm models, the present invention can export all types of robot arm models into URDF files by using the robot arm model data and the URDF plug-in of SolidWorks software, so that they can be simulated and verified in the simulation system.

[0088] 2. Optimize the motion control of the robot arm: Combine the forward and inverse kinematics algorithms and point-to-point interpolation motion algorithms provided by the Bullet engine to make the movement of the robot arm smoother and more natural. Combined with the joint parameter settings in the robot arm URDF file, the motion logic of the simulated robot arm is more similar to the motion process of the real robot arm, preventing unnecessary risks caused by the difference between the simulation results and the real output during the real motion process.

[0089] 3. Automatically generate the spraying trajectory of the robot arm: The present invention generates a spraying trajectory adapted to the current steel structure based on the point cloud data of the steel structure through process parameters and trajectory algorithms, which not only improves the working efficiency of the robot arm, but also greatly reduces the cost of redesigning the trajectory according to different steel structures.

[0090] 4. Improve the safety of the robot arm: After the trajectory algorithm calculates the spraying trajectory, the trajectory list will be sent to the simulation system for simulation verification. In the simulation system, if there is no collision or problem during the trajectory operation, the trajectory list will be sent to the real robot. At the same time, the system will automatically generate transition points before the robot arm spraying operation to ensure that the robot can avoid obstacles such as itself, workpieces and transport vehicles during movement, and avoid potential collision risks, thereby greatly improving the safety of the robot arm operation and reducing the cost and risk of actual operation.

[0091] The best embodiment for carrying out the present invention will be described in more detail below with reference to the accompanying drawings so that the features and advantages of the present invention can be easily understood. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings of the embodiments of the present invention, wherein the drawings are only used to illustrate some embodiments of the present invention, but not to limit all embodiments of the present invention thereto.

[0093] Figure 1 A schematic diagram of a flow chart of a simulation method of a spraying robot arm based on a Bullet engine according to an embodiment of the present invention;

[0094] Figure 2 A schematic diagram of a process for constructing a complex motion simulation model of a multi-degree-of-freedom robotic arm according to an embodiment of the present invention;

[0095] Figure 3 The figure is a schematic diagram of a process of calculating a spraying trajectory from point cloud data of a steel structure according to an embodiment of the present invention. DETAILED DESCRIPTION

[0096] In order to make the purpose, technical solution and advantages of the technical solution of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described in conjunction with the drawings of specific embodiments of the present invention. The same figure marks in the drawings represent the same parts. It should be noted that the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0097] Unless otherwise defined, the technical terms or scientific terms used herein shall have the usual meanings understood by persons of ordinary skill in the field to which the present invention belongs. The words "first", "second" and similar words used in the patent application specification and claims of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "one" do not necessarily indicate a quantity limitation. Words such as "include" or "comprise" mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0098] According to a first aspect of the present invention, a simulation method for a spraying robot arm based on a Bullet engine is as follows: Figure 1 As shown, the following steps are included:

[0099] S10: Figure 2 As shown, SolidWorks software combined with URDF plug-in is used to perform 3D modeling based on the spraying robot arm model data, basic axes and coordinate systems are established at the joints of the spraying robot arm, and corresponding kinematic parameters are set, and the URDF file of the spraying robot arm is exported;

[0100] Specifically, in order to improve the operating efficiency and safety of the robot arm in the actual working scene, it is particularly important to obtain and build a motion simulation model of the robot arm. By combining the three-dimensional model of the object in the working scene and building a realistic simulation working environment, the working conditions of the robot arm in actual operation can be more comprehensively simulated. With the help of the simulation environment, potential problems can be discovered in advance and the operation process can be optimized, thereby reducing the risks and losses that may be caused in actual operations. In this way, not only can work efficiency be improved, but also failures and safety hazards caused by incorrect operation or improper design during the operation of the robot arm can be significantly reduced.

[0101] like Figure 2 As shown in the figure, first, according to the model of the selected robot arm, the corresponding 3D model file is obtained. Subsequently, these 3D model files are imported into SolidWorks software to be further converted into URDF files for control and motion simulation in the simulation system. URDF file is a format based on XML syntax, which is specifically used to describe the geometry and motion information of the robot structure. For the robot arm, the URDF file needs to clearly describe the relative relationship between each link, the joint type and its parameters, so as to provide a basis for the subsequent call of the Bullet engine control function to achieve precise control of the robot arm movement.

[0102] A robotic arm is usually composed of several links and joints between them. Therefore, when generating a URDF file in SolidWorks, you must first insert reference geometry points based on the geometric features of each link. These points will serve as the origin reference position of the coordinate system. The type of joint between each link also needs to be defined according to its motion mode. Common joint types include revolute joints and sliding joints. For revolute joints, you also need to insert a reference axis in the model as the rotation axis of the joint. According to the inserted coordinate points and reference axis, a coordinate system is established for each link. In order to ensure that the robotic arm can be displayed and moved correctly in the simulation system, special attention should be paid to the direction of the coordinate axis when establishing the coordinate system of each link. The coordinate system Z axis of the base of the robotic arm must be consistent with the upright direction of the robotic arm, otherwise the robotic arm may be inverted in the simulation system. In addition, the coordinate system Z axis of all links should be parallel to the rotation axis of the corresponding joint to ensure that the motion in the simulation conforms to the actual motion mode of the robotic arm.

[0103] After completing the establishment of the rotation axis and coordinate system, we can start the URDF plug-in and start configuring the relevant parameters of the robot. First, configure the base part of the robot. In this step, you need to set a name for the base link and set its coordinate system to the previously created base coordinate system. In the components of the link, select the part of the model that represents the base and set the number of sub-links to 1. Then, continue to configure the sub-link parameters of the base. When configuring the sub-link, in addition to the same parameter settings as the base link, you also need to set a name for the joint between the base and the sub-link. The joint motion type and the reference coordinate axis of the joint motion also need to be configured accordingly. Then, according to the number of links and joints of the robot, gradually set the parameters for each link and the corresponding joint. When all the links and joints are set, enter the export interface of the URDF file. When exporting the URDF file, according to different models of robot arms, further set the key parameters such as the maximum and minimum rotation angles, torque, and rotation speed for each joint. After completing all parameter settings, you can export the URDF file of the robot.

[0104] In addition, it is important to note that when the URDF file is imported into the simulation system, the position of each link of the robot arm is determined by the center coordinates of the link. This mechanism will cause the end position of the robot arm model to be aligned with the center coordinates of the last link, rather than the actual end of the robot arm, when it is generated in the simulation system. At the same time, the position coordinates of the entire robot arm will also be based on the center coordinates of the base, rather than the position coordinates of the lowest end of the robot arm. Therefore, in order to ensure that the position coordinates and end coordinates of the robot arm are more accurate in the simulation system, we need to add two virtual links to the generated URDF file. Specifically, the child link of the first virtual link is set to the base, and its joint motion type should be fixed. The parent link of the second virtual link is the last link of the robot arm, and the joint motion type is also fixed. There is no need to set parameters such as collision detection and material appearance for these two virtual links. In this way, the position and end coordinates of the robot arm in the simulation environment can be ensured to be more accurate, thereby optimizing the simulation effect.

[0105] Through the above steps, the three-dimensional model of the robotic arm is accurately converted into a URDF file, which not only lays the foundation for the robotic arm control in the simulation system, but also provides more accurate parameter support for the subsequent call of the control function of the simulation engine, ensuring that the motion trajectory and operation stability during the simulation process can be truly simulated.

[0106] S20: Based on the Bullet engine, the forward kinematics equations of the motion angles of each joint of the spraying robot arm are calculated according to the coordinates and posture of the end of the spraying robot arm, or the end posture of the spraying robot arm is calculated through the motion angles of each joint of the spraying robot arm, and the inverse kinematics equations of controlling the spraying robot arm to move to the specified position are used to realize the motion control of the spraying robot arm in the simulation environment;

[0107] S30: On the basis of the Bullet engine, a variety of working scenes of the spraying robot arm are built and created in the STL file format, the corresponding working environment type is selected and the STL format file of the working environment type is added to the model of the spraying robot arm; specifically, the present invention designs an STL file import module on the basis of the Bullet engine, which can efficiently create the required working scene in the STL file format. This process not only involves the geometric shape of the scene, but also includes the coordinate information of the specific position of the object in the scene. This method makes it more intuitive and convenient to build complex working scenes in the simulation system. After the scene is built, the scene can be saved in the JSON file format, which is convenient for importing into the simulation system in subsequent use. In this way, the previously built scene can be quickly loaded, thereby saving time and energy. In addition, the simulation system also supports the direct import of the built scene file, providing more flexibility. Whether it is reusing the previous scene or quickly building a working environment in a new project, the STL file import module can build and manage the simulation work scene in an efficient and flexible manner.

[0108] S40: Scan the steel structure and form point cloud data, and generate the spraying trajectory of the spraying robot arm in combination with the spraying process parameters of the steel structure, and import the spraying trajectory into the spraying robot arm model in the simulation environment for verification. If no error occurs during the simulation verification process, save the spraying trajectory data; if an error occurs during the simulation process, perform corresponding correction and optimization processing according to the error type until the simulation is correct.

[0109] The simulation method can adapt to various robot arm models. By using the robot arm model data and the URDF plug-in of SolidWorks software, the present invention can export all types of robot arm models into URDF files so that they can be simulated and verified in the simulation system.

[0110] This simulation method can optimize the motion control of the robotic arm: combined with the forward and inverse kinematics algorithms and point-to-point interpolation motion algorithms provided by the Bullet engine, the movement of the robotic arm is smoother and more natural. Combined with the joint parameter settings in the robotic arm URDF file, the motion logic of the simulated robotic arm is more similar to the motion process of the real robotic arm, preventing unnecessary risks caused by the difference between the simulation results and the real output during the real motion process.

[0111] This simulation method can automatically generate the spraying trajectory of the robot arm: the present invention generates a spraying trajectory adapted to the current steel structure based on the point cloud data of the steel structure through process parameters and trajectory algorithms, which not only improves the working efficiency of the robot arm, but also greatly reduces the cost of redesigning the trajectory according to different steel structure needs.

[0112] This simulation method can improve the safety of the robot arm: after the trajectory algorithm calculates the spraying trajectory, the trajectory list will be sent to the simulation system for simulation verification. In the simulation system, if there are no collisions or problems during the trajectory operation, the trajectory list will be sent to the real robot. At the same time, the system will automatically generate transition points before the robot arm sprays, ensuring that the robot can avoid obstacles such as itself, workpieces and transport vehicles during movement, avoiding potential collision risks, thereby greatly improving the safety of the robot arm operation and reducing the cost and risk of actual operation.

[0113] In one example of the present invention, Figure 2 As shown, the step S10 includes the following:

[0114] By using SolidWorks software, combined with the URDF plug-in, users can efficiently model robot arm model files from various manufacturers. In this process, it is first necessary to establish the basic axis and coordinate system for each joint of the robot arm. This step is the basis for ensuring that the joints of the robot arm can move correctly. Next, it is necessary to set parameters such as the rotation angle, speed, and torque of the joint. These settings not only affect the motion performance of the robot arm, but also its efficiency and safety in actual operation. Through precise parameter configuration, it can be ensured that the robot arm exhibits motion characteristics consistent with actual operation in the simulation environment.

[0115] After completing the above steps, the URDF (Uniform Robot Description Format) file of the robot arm is exported. The generation of this file is a key link in the entire modeling process, and it provides necessary data support for subsequent simulation experiments. Finally, the generated URDF file is imported into the simulation platform to conduct various simulation experiments. Through the combination of SolidWorks software and URDF plug-in, users can systematically build a robot arm model and complete the entire process from modeling to simulation experiments, laying a solid foundation for the practical application of the robot arm.

[0116] In an example of the present invention, step S20 includes the following steps:

[0117] S21: Construct the kinematic model of the robot arm in the simulation system. The spray robot arm consists of serially connected joints and links. For a robot arm with n degrees of freedom, it has n joints, the joints are numbered from 1 to n, there are n+1 links, the links are numbered from 0 to n, link 0 is the base of the robot arm, which is generally fixed, link n has an end effector, and joint i connects link i and link i-1; based on the DH (Denavit-Hartenberg) parameters, define the transformation matrix T of each joint i :

[0118]

[0119] Among them, a i It is along X i Axis, from Z i Move to Z i+1 The distance, i.e., the abstract length of connecting rod i; θ i It is around Z i Axis, from X i-1 Rotate to X i The angle between connecting rod i-1 and connecting rod i; α i From Z i Rotate to Z i+1 Angle, the axis of rotation is X i , that is, the angle between connecting rod i and connecting rod i+1; d i It is along Z i Axis, from X i-1 Move to X i distance;

[0120] S22: Solve the forward kinematics equation to calculate the end pose or solve the inverse kinematics equation to calculate the joint angle;

[0121] Solving the forward kinematics equations to compute the end pose involves:

[0122] The end pose is calculated by multiplying the transformation matrices of all joints:

[0123] T(q)=T1·T2……T n

[0124] The end position and posture are separated into:

[0125]

[0126] Among them, q is the motion angle of the joint, T(q) is the solution of positive kinematics, that is, the end position, p(q) is the end coordinate, and p(q) is the end posture.

[0127] Solving the inverse kinematics equations to calculate joint angles includes:

[0128] Calculate the current end pose error:

[0129] Position error:

[0130] Δp=p d -p(q)

[0131] Attitude error:

[0132] ΔR=R d ·R(q) T

[0133] By converting ΔR into a rotation vector Δr, we get:

[0134] Δr=logΔR

[0135] Comprehensive error:

[0136] ΔT=[Δp,Δr]

[0137] Compute the Jacobian matrix:

[0138] The Jacobian matrix J(q) represents the partial derivatives of the end pose with respect to the joint angle:

[0139]

[0140] Joint variable update:

[0141] Update the joint angles by solving linear equations:

[0142] Δq=J(q) -1 ΔT

[0143] Update joint angles:

[0144] q new =q+Δq

[0145] Check convergence:

[0146] If the error ||ΔT||<ε, the iteration is exited, and the joint angle solved by inverse kinematics is q new ;

[0147] S23: When solving the inverse kinematics equation to calculate the joint angle, the joint sequence number set in the URDF file is used to control the movement of each joint of the spray robot arm to the joint angle solved by the inverse kinematics, thereby realizing the motion control of the robot arm in the simulation system.

[0148] In one example of the present invention, in step S22, in the process of solving the joint motion angle by the inverse kinematics algorithm according to the pose information of the target point, it also includes: combining the interpolation algorithm to generate a series of discrete intermediate points between the starting point and the target point, so that the end effector of the robot arm moves along a straight line, and the specific steps are as follows:

[0149] S221: First, interpolate the coordinates of the two points. For linear motion, use the parameter λ∈[0,1] to represent the ratio from the starting point to the end point:

[0150] p(λ)=(1-λ)·p s +λ·p e

[0151] Among them, p s is the coordinate information of the starting point, p e is the coordinate information of the end point and the coordinate information of the interpolation point p;

[0152] According to the number of discretized points N from the starting point to the end point, calculate λ i =i / N, generate position sequence:

[0153] p i = p(λ i )

[0154] S222: Use the quaternion interpolation algorithm for the two-point posture, and convert the rotation matrix R of the starting point and the end point s and R e Convert to quaternion q s and q e , use the spherical linear interpolation algorithm to calculate the quaternion q of the interpolation point i , the quaternion q of the interpolation point i Then convert back to the terminal posture R i ; Among them, the quaternion q i The expression is as follows:

[0155]

[0156] Where θ = cos -1 (q s ·q e ) is the angle of the quaternion;

[0157] S223: For each interpolation point coordinate P i and the terminal posture R i , combined to generate the interpolation point pose P i =[P i ,R i ], and save the result in the interpolation point sequence P;

[0158] S224: Calculate the joint angle of the robot arm for each interpolation point through the inverse kinematics algorithm, and gradually send joint angle instructions according to the interpolation results to control the movement of the robot arm to achieve linear movement of the robot arm in the simulation system.

[0159] In the simulation system, the joint motion angle is solved by the inverse kinematics algorithm according to the posture information of the target point. However, only the target posture information will cause the movement process of the robot arm to the target point not necessarily to be a linear motion, which will make the motion trajectory uncontrollable. Therefore, the present invention combines the interpolation algorithm to generate a series of discrete intermediate points between the starting point and the target point, so that the end effector of the robot arm moves in a straight line.

[0160] First, the robot's posture information in the trajectory list is obtained, and the motion angles of each joint of the robot are solved based on the Bullet engine and the inverse kinematics algorithm. Then, based on the Bullet engine and the joint parameters configured in the robot's URDF file, the motors of each joint of the robot are controlled to move to the joint angles solved by inverse kinematics according to the serial numbers of each joint setting in the URDF file. In the simulation environment, the joint motion angles are inversely solved based on the posture information of the two points, but there are only two-point joint angles, which will cause the robot's movement between the two points to be not necessarily a straight line. Therefore, the interpolation method is needed to subdivide the distance between the two points into multiple two-point movements. By moving step by step, the robot can smoothly complete the movement from one point to another on the simulation platform, avoiding abrupt changes that cause problems in the simulated trajectory path.

[0161] In one example of the present invention, Figure 3 As shown, the step S40 specifically includes the following steps:

[0162] S41: acquiring point cloud data of each angle of the steel structure by a multi-angle scanner, and obtaining the overall point cloud data of the steel structure by point cloud registration;

[0163] S42: dividing the obtained overall point cloud data of the steel structure into cross-section data and vertical plate data, and pre-processing the cross-section data and the vertical plate data;

[0164] S43: Calculate the spraying range of the spraying robot arm in combination with the process parameters of the steel structure, and calculate the number of spraying tracks according to the spraying range of the spraying robot arm;

[0165] S44: Calculate characteristic points of the steel structure according to the number of spraying trajectories and the spraying range, and convert the characteristic points of the steel structure into the spatial position and spraying posture of the robot arm corresponding to each trajectory;

[0166] S45: Import the final spraying trajectory of the spraying robot arm into the spraying robot arm model in the simulation environment for verification.

[0167] Specifically, the steel structure to be sprayed is scanned in detail by a multi-angle scanner, and the corresponding three-dimensional data files are generated. First, multiple scanning devices are set at different angles of the steel structure to fully cover all parts of the entire structure. During the scanning process, these devices will generate multiple sets of point cloud files, each of which corresponds to the scanning results at different angles. Subsequently, the point cloud data from different angles are registered into the overall point cloud model of the steel structure through the point cloud registration algorithm. This step ensures that the geometric form of the steel structure can be restored completely and accurately. After obtaining the complete point cloud data, the simulation system will further apply a special section and vertical plate processing algorithm to extract the section data and vertical plate data from it. When the simulation system receives the section data and vertical plate data generated by the visual algorithm, it will first pre-process these data. The main goal of the pre-processing is to extract accurate coordinate information that can be used for further calculations. These coordinate information will be combined with the set spraying parameters, such as the height between the nozzle and the plate surface, the angle between the nozzle and the plate surface, and the coverage of the nozzle. Combined with these parameters, the system calculates the best spraying trajectory through the trajectory algorithm. The trajectory includes the spatial position, motion path and spraying posture of the nozzle during the spraying process. The generated spraying trajectory will be verified on the simulation system to ensure its rationality and accuracy. After the simulation verification, the system will save the final generated spraying trajectory into the trajectory list. This trajectory list will be used for subsequent actual spraying operations as the basis for controlling the robot arm to perform the spraying task.

[0168] In an example of the present invention, in step S41, obtaining the overall point cloud data of the steel structure through point cloud registration includes the following steps:

[0169] S411: Performing rough registration, firstly performing fast point feature histogram feature extraction on the source point cloud and the target point cloud respectively, obtaining the FPFH feature vector of each point, then using the feature matching algorithm to find similar feature vectors in the source point cloud and the target point cloud, and calculating the distance between the two FPFH feature vectors, wherein if the distance is small enough, they are considered to be corresponding point pairs; after finding a set of initial feature matching pairs, using the RANSAC algorithm to estimate the initial transformation matrix, wherein the RANSAC algorithm finds the optimal transformation matrix through random sampling and iterative optimization, and finally obtains the rotation matrix R and translation vector T of rough registration;

[0170] S412: Perform fine registration, use the initial transformation matrix obtained by coarse registration, and use the ICP algorithm to iteratively optimize to obtain a more accurate solution. The ICP algorithm calculates the distance between the corresponding points of the source point cloud and the target point cloud, constructs the rotation and translation matrix RT, transforms the source point cloud through RT, and calculates the mean square error after the transformation; if the mean square error meets the threshold condition, the algorithm ends, otherwise it continues to iterate until the error meets the threshold condition or the number of iterations is terminated, and finally the overall point cloud data of the steel structure is obtained.

[0171] Specifically, the robot arm motion trajectory process is obtained by combining the steel structure point cloud data through the trajectory algorithm as follows: Figure 3 As shown in the figure, multi-angle point cloud data is obtained by a multi-angle scanner, and then the multi-angle point cloud files are aligned. There are two steps. The first step is to perform rough alignment. Rough alignment is to perform point cloud alignment when the source point cloud and the target point cloud are completely unaware of any relative initial position. Generally, a method based on local feature description is used for rough alignment. First, FPFH (Fast Point Feature Histogram) feature extraction is performed on the source point cloud and the target point cloud respectively to obtain the FPFH feature vector of each point, which describes the local geometric structure of the point. Then, a feature matching algorithm is used to find similar feature vectors in the source point cloud and the target point cloud. The commonly used measurement method is Euclidean distance, that is, calculating the distance between two FPFH feature vectors. If the distance is small enough, they are considered to be corresponding point pairs. In the feature matching process, Euclidean distance is used to measure the similarity of two feature vectors. After finding a set of initial feature matching pairs, the RANSAC (Random Sampling Consensus) algorithm is used to estimate the initial transformation matrix. The RANSAC algorithm finds the optimal transformation matrix through random sampling and iterative optimization. Finally, the rotation matrix R and translation vector T of rough alignment are obtained. The second step is fine registration, which is based on the rough registration to perform a more accurate and detailed registration. Using the initial transformation matrix obtained from the rough registration, the ICP (Iterative Closest Point) algorithm is used to iteratively optimize and obtain a more accurate solution. The ICP algorithm calculates the distance between the corresponding points of the source point cloud and the target point cloud, constructs the rotation and translation matrix RT, transforms the source point cloud through RT, and calculates the mean square error after the transformation. If the mean square error meets the threshold condition, the algorithm ends. Otherwise, the iteration continues until the error meets the threshold condition or the number of iterations ends. The overall point cloud data of the steel structure can be obtained through the registration calculation of the two stages of coarse registration and fine registration.

[0172] In one example of the present invention, in step S42, dividing the obtained overall point cloud data of the steel structure into cross-section data and vertical plate data specifically includes the following steps:

[0173] S421: plane cutting the overall point cloud data of the steel structure along its length extension direction in units of K millimeters to obtain multiple three-dimensional cutting sections;

[0174] S422: According to the type of steel structure, it is divided into two categories: H-shaped steel and square steel. For the cross-section processing of H-shaped steel, the web is used as the symmetry center to divide the cross-section into two independent characteristic structures on the left and right; for the cross-section of square steel, each side of the square cross-section is used as the dividing plane to divide the cross-section into four independent characteristic structures;

[0175] S423: The characteristic structure divided by the cutting plane is processed as a basic unit, and its envelope data is extracted and described. Finally, the point cloud data of the entire steel structure is further subdivided into cross-section data and facade data.

[0176] Specifically, the overall point cloud data of the steel structure is divided into cross-section data and vertical plate data. First, the overall point cloud data of the steel structure is plane-cut in units of K millimeters along its length extension direction to obtain multiple three-dimensional cutting sections. According to the type of steel structure, the steel structure can be divided into two categories: H-shaped steel and square steel. For the cross-section processing of H-shaped steel, the web is used as the symmetry center to divide the cross-section into two independent feature structures on the left and right; and for the cross-section of square steel, each side of the square cross-section is used as the dividing plane to divide the cross-section into four independent feature structures. Each cutting plane will be processed based on the above-mentioned "single-direction feature block" as the basic unit to extract and describe its envelope data. Through this cutting process, the point cloud data of the steel structure can be refined so that various feature structures can be expressed independently in three-dimensional space. Finally, the point cloud data of the entire steel structure is further subdivided into cross-section data and vertical plate data. The obtained cross-section and vertical plate data packets contain clear start and end marks to ensure the integrity and identifiability of the data. According to the type of steel structure identified, these data can be divided into: H-shaped steel and square steel. Each steel structure corresponds to a sequential number to facilitate tracking and reference in data management and analysis. The types of data are mainly divided into section data and facade data, and each data type has a corresponding number of data. Section data is a further supplement to the common data shared by the two types of data. Each section data contains the following additional information to identify the sequential number of the data, the X-axis coordinate value of each section data, the number of normal vectors, the number of envelope points, the center point coordinates of each section data, and the single-direction feature module data: This part of the data is different from H-shaped steel and square steel, and the number of each single-direction feature module is 2 or 4. The serial number of each module is divided into 0, 1, 2 and 0, 1, 2, 3. In addition, the single-direction feature module data also includes: the normal vector coordinates of the first single-direction feature module and the envelope point coordinates of each single-direction feature module. The facade data has the number of height points on the basis of the section data, and the specific height value of the height point corresponding to the facade data type.

[0177] In an example of the present invention, in step S42, preprocessing the cross-section data and the vertical plate data specifically includes the following steps:

[0178] The point cloud coordinates of the steel structure are converted from the original reference coordinate system to the world coordinate system of the simulation system through affine transformation;

[0179] The point cloud data of the steel structure is divided into N groups of data according to the working range of the robot arm, and the length of each group of data is the working range of the robot arm;

[0180] For a single section data and vertical plate data, find the maximum and minimum values ​​in the Z-axis and Y-axis directions. These four points will be used as the starting and end points of the subsequent trajectory planning, providing necessary reference information for trajectory calculation;

[0181] The pre-processed section data and facade data are packaged into a TXT file for subsequent processing.

[0182] First, the cross-section data and vertical plate data of the steel structure are preprocessed to extract coordinate information for subsequent trajectory calculation. Since the point cloud data coordinates of the steel structure are generated based on the reference coordinate system during scanning, in order to achieve more accurate positioning in the simulation system, the point cloud coordinates need to be converted to the coordinate system. Specifically, the point cloud coordinates of the steel structure are converted from the original reference coordinate system to the world coordinate system of the simulation system through affine transformation to ensure a unified spatial reference in the simulation environment. After completing the coordinate system conversion, the point cloud data of the steel structure is divided into N groups of data according to the working range of the robot arm, and the length of each group of data is the working range of the robot arm. For a single cross-section data and vertical plate data, find the maximum and minimum values ​​in the Z-axis and Y-axis directions. These four points will be used as the starting and end points of the subsequent trajectory planning to provide necessary reference information for trajectory calculation. Finally, the preprocessed cross-section data and vertical plate data are packaged into a TXT file for subsequent processing.

[0183] In an example of the present invention, in step S43, the spraying range of the spraying robot arm is calculated in combination with the process parameters of the steel structure, wherein the spraying range calculation formula is as follows:

[0184]

[0185] Among them, S is the spraying range, D is the height of the nozzle from the board surface, and A is the nozzle angle.

[0186] In one example of the present invention, in step S43, calculating the number of spraying tracks according to the spraying range of the spraying robot arm includes the following steps:

[0187] S431: Based on the spraying margin in the process parameters and the overlap of the two spraying trajectories, start calculating the first starting point of each set of data; wherein the spraying margin is defined as the increase or decrease of the spraying amount of the next set after the robot arm completes each set of spraying tasks, which means that the robot arm may spray more or less between different groups; the overlap of the two spraying trajectories is deducted from the overall spraying range, and the actual spacing between the trajectories is calculated to further determine the spraying path of the next trajectory;

[0188] S432: After determining the coordinates of the first point, the remaining spraying range is calculated through the first point and the range of each set of point cloud data;

[0189] S433: Divide the remaining spraying range by the distance between the two tracks to calculate how many tracks are needed to spray the area;

[0190] S434: Multiply the number of trajectories by the distance between the trajectories and subtract the remaining spraying range to obtain the remainder of the current set of spraying data, which will be used as a reference for calculating the starting point of the next set of data.

[0191] In one example of the present invention, in step S44, the characteristic points of the steel structure are calculated according to the number of spraying trajectories and the spraying range, and the characteristic points of the steel structure are converted into the spatial position of the robot arm and the spraying posture corresponding to each trajectory, which specifically includes the following steps:

[0192] S441: The first starting point position plus N multiplied by the distance between the two spraying tracks is used to calculate the feature point position of the Nth track;

[0193] S442: According to the characteristic points of the Nth trajectory, the cross-section data and the vertical plate data within the spraying range of each trajectory are traversed, and the starting point, the ending point, the highest point and the lowest point of each spraying trajectory are determined according to the four key points obtained after preprocessing;

[0194] S443: After obtaining the trajectory spraying coordinates, they need to be converted into the spatial position (x, y, z) of the robot arm and the corresponding spraying posture (Rx, Ry, Rz);

[0195] S444: Save the spraying robot arm posture data of each spraying trajectory into a list to form complete spraying trajectory coordinate data.

[0196] In one example of the present invention, in step S45, importing the final spraying trajectory of the spraying robot arm into the spraying robot arm model in the simulation environment for verification includes:

[0197] Choose whether to optimize the trajectory or directly output the trajectory list based on the results of the simulation verification. If no errors occur during the simulation verification, the system will save the trajectory list so that it can be sent to the actual robot for spraying operations later. However, if an error occurs during the simulation, the system will handle it accordingly according to the type of error. For example, if the error is due to exceeding the range of motion of the external axis of the robot arm or a collision, the trajectory will be judged as unavailable, the system will recalculate a new spraying trajectory, and perform simulation verification again. On the other hand, if the simulation verification does not exceed the range of motion of the external axis but a collision occurs, the system will further check the distance between the nozzle and the board surface. If this distance does not exceed the set range, the system will adjust the distance of the external axis and recalculate to generate a new spraying trajectory. On the contrary, if it is found that the distance between the nozzle and the board surface exceeds the set range, the system will modify the distance between the nozzle and the board surface, and then calculate again to generate a new spraying trajectory. This process will be repeated until the spraying trajectory is correct in the simulation verification.

[0198] In an example of the present invention, after step S20, the method further includes: based on the Bullet engine, using a physics-based collision detection algorithm, adding a bounding box to the spray robot arm model to determine whether the generated spray trajectory will cause a collision between the robot arm and the object.

[0199] Based on the Bullet engine, the present invention uses a physics-based collision detection algorithm to add a bounding box to the object model. This process is intended to simulate the interaction and collision between objects, thereby achieving a more realistic physical simulation. The simulation system performs collision detection for the bounding box of each object. By detecting the bounding box, the system can determine whether the generated spraying trajectory will cause a collision between the robot arm and the object. This judgment not only helps to identify potential collision risks, but also provides a basis for subsequent path optimization. Based on the results of the collision detection, the generated spraying path can be optimized to ensure that the real robot arm can effectively avoid obstacles such as itself, workpieces and transport vehicles during movement. This optimization measure is intended to reduce the possibility of collisions and enhance the safety and efficiency of operations.

[0200] Another object of the present invention is to provide a simulation system for a spraying robot arm based on a Bullet engine, comprising:

[0201] A robot arm model creation module is configured to use SolidWorks software in combination with a URDF plug-in to perform three-dimensional modeling based on the spraying robot arm model data, establish basic axes and coordinate systems at the joints of the spraying robot arm, set corresponding kinematic parameters, and export a URDF file of the spraying robot arm;

[0202] The robot control module is configured to calculate the forward kinematics equation of the motion angle of each joint of the spraying robot arm based on the Bullet engine, or calculate the end position of the spraying robot arm through the motion angle of each joint of the spraying robot arm, and control the inverse kinematics equation of the spraying robot arm to move to the specified position, so as to realize the motion control of the spraying robot arm in the simulation environment; specifically, it is used to control the movement of the robot arm. The robot to be driven in the scene can be selected, and different robots can be operated separately; the external axis drive of the selected robot can be driven by raising or lowering the robot body to avoid moving to the singular point of the robot working range, so as to optimize the trajectory path; the robot arm joint data is obtained and displayed, and the end position information of the robot arm is obtained according to different reference coordinate systems (world coordinate system or robot arm coordinate system) or the robot arm movement control is performed by setting the end position of the robot arm. The end position coordinates of the robot arm or the robot arm joint angle parameters can also be set to control the robot arm to run to the specified position.

[0203] The work scene import module is configured to build a variety of work scenes of the spray robot arm based on the Bullet engine and create them in STL file format, select the corresponding work environment type and add the STL format file of the work environment type to the model of the spray robot arm; specifically, it is used to browse the model file directory and replace some models according to the specific work scene. The built work scene can be exported and saved as a configuration file, or the corresponding configuration file can be selected for reading and importing. After successfully importing the work scene, the system allows some entity models in the scene to be replaced, which is convenient for dynamic adjustment of the model according to actual needs. After the replacement is completed, the updated work environment can be generated by refreshing the scene to realize the presentation of the replaced model. This module not only improves the flexibility of model management, but also makes the configuration and loading of models in different work scenes more efficient; select the entity model in the work scene, and perform model operations on the selected model, such as reading the current position, moving the model, adding collision volume, and removing collision volume. . After loading the model, the system will automatically read the current position of the selected model, and you can select the model to replace its spatial coordinates (x, y, z) and the Euler angle (r) of the posture. x ,r y ,r z ) is displayed in the parameter list. After resetting the current model position as needed, click Model Move to manually adjust the model's current position.

[0204] The spraying trajectory calculation module is configured to scan the steel structure and form point cloud data, and generate the spraying trajectory of the spraying robot arm in combination with the spraying process parameters of the steel structure, and import the spraying trajectory into the spraying robot arm model in the simulation environment for verification, wherein, if no error occurs during the simulation verification process, the spraying trajectory data is saved; if an error occurs during the simulation process, the corresponding correction and optimization processing is performed according to the error type until the simulation is correct. Specifically, it is used to generate the spraying trajectory of the robot arm movement from the point cloud data and trajectory algorithm of the steel structure. The scanned steel structure data is preprocessed and packaged into a TXT file and saved in a local directory, and then the motion trajectory is generated by the trajectory algorithm according to the vertical plate data or the cross-section data, and simulated motion is performed in the simulation scene. In order to facilitate the intuitive viewing of the trajectory route in the simulation scene, the Bullet engine is used to add trajectory tracking to the robot arm, leaving a running trace at the end of the robot arm. After the robot arm moves correctly in the scene, the trajectory list is saved and displayed in the trajectory list window. You can also modify the process parameters or the external axis of the robot arm, click the trajectory optimization button, and recalculate and verify the trajectory.

[0205] The simulation system can adapt to various mechanical arm models. By using the mechanical arm model data and the URDF plug-in of SolidWorks software, the present invention can export all types of mechanical arm models into URDF files so that they can be simulated and verified in the simulation system.

[0206] The simulation system can optimize the motion control of the robotic arm: combining the forward and inverse kinematics algorithms and point-to-point interpolation motion algorithms provided by the Bullet engine makes the movement of the robotic arm smoother and more natural. Combined with the joint parameter settings in the robotic arm URDF file, the motion logic of the simulated robotic arm is more similar to the motion process of the real robotic arm, preventing unnecessary risks caused by differences between simulation results and real outputs during real motion.

[0207] The simulation system can automatically generate the spraying trajectory of the robot arm: the present invention generates a spraying trajectory adapted to the current steel structure based on the point cloud data of the steel structure through process parameters and trajectory algorithms, which not only improves the working efficiency of the robot arm, but also greatly reduces the cost of redesigning the trajectory according to different steel structure needs.

[0208] The simulation system can improve the safety of the robot arm: after the trajectory algorithm calculates the spraying trajectory, the trajectory list will be sent to the simulation system for simulation verification. In the simulation system, if there are no collisions or problems during the trajectory operation, the trajectory list will be sent to the real robot. At the same time, the system will automatically generate transition points before the robot arm sprays, ensuring that the robot can avoid obstacles such as itself, workpieces and transport vehicles during movement, avoiding potential collision risks, thereby greatly improving the safety of the robot arm operation and reducing the cost and risk of actual operation.

[0209] In one example of the present invention, it also includes: a process module configured to input process parameters, including the height of the nozzle from the board, the overlap of the spraying trajectory, the angle between the nozzle and the board surface, and the nozzle angle. The process parameters can be saved and loaded, and the margin size obtained in the trajectory calculation can be displayed at the same time.

[0210] In one example of the present invention, it also includes: a viewing angle control module, which is configured to be used to display the constructed simulation scene. The viewing angle state can be unlocked to control the selection of a suitable angle to observe the spraying operation process of the robotic arm. The point selection function can also be turned on in the state of locking the viewing angle to obtain the position of the selected point in the world coordinate system. During the movement of the robotic arm, if it collides with other rigid objects, a collision point will be left at the collision position and displayed in the display interface to facilitate trajectory optimization and safety inspection.

[0211] The exemplary implementation methods of the simulation method and system of the spray robot arm based on the Bullet engine proposed in the present invention are described in detail above with reference to the preferred embodiments. However, those skilled in the art will appreciate that, without departing from the concept of the present invention, various modifications and variations can be made to the above-mentioned specific embodiments, and various technical features and structures proposed in the present invention can be combined in various ways without exceeding the protection scope of the present invention, which is determined by the attached claims.

Claims

1. A simulation method for a spraying robot arm based on the Bullet engine, characterized in that: The steps include: S10: Use SolidWorks software combined with URDF plug-in to perform 3D modeling based on the spraying robot arm model data, establish basic axes and coordinate systems at the joints of the spraying robot arm, set corresponding kinematic parameters, and export the URDF file of the spraying robot arm; S20: Based on the Bullet engine, the forward kinematics equations of the motion angles of each joint of the spraying robot arm are calculated according to the coordinates and posture of the end of the spraying robot arm, or the end posture of the spraying robot arm is calculated through the motion angles of each joint of the spraying robot arm, and the inverse kinematics equations of controlling the spraying robot arm to move to the specified position are used to realize the motion control of the spraying robot arm in the simulation environment; S30: Based on the Bullet engine, build various working scenes of the spraying robot arm and create them in STL file format, select the corresponding working environment type and add the STL format file of the working environment type to the model of the spraying robot arm; S40: Scan the steel structure and generate point cloud data, generate a spraying trajectory of the spraying robot arm in combination with the spraying process parameters of the steel structure, and import the spraying trajectory into the spraying robot arm model in the simulation environment for verification, wherein if no error occurs during the simulation verification process, save the spraying trajectory data; If an error occurs during the simulation, perform corresponding corrections and optimization according to the error type until the simulation is correct.

2. The simulation method of the spraying robot arm based on the Bullet engine according to claim 1, characterized in that: The step S20 comprises the following steps: S21: Construct the kinematic model of the robot arm in the simulation system. The spray robot arm consists of serially connected joints and links. For a robot arm with n degrees of freedom, it has n joints, the joints are numbered from 1 to n, there are n+1 links, the links are numbered from 0 to n, link 0 is the base of the robot arm, link n has an end effector, and joint i connects link i and link i-1; Based on the DH parameters, define the transformation matrix T of each joint i : Among them, a i It is along X i Axis, from Z i Move to Z i+1 The distance, i.e., the abstract length of connecting rod i; θ i It is around Z i Axis, from X i-1 Rotate to X i The angle between connecting rod i-1 and connecting rod i; α i From Z i Rotate to Z i+1 Angle, the axis of rotation is X i , that is, the angle between connecting rod i and connecting rod i+1; d i It is along Z i Axis, from X i-1 Move to X i distance; S22: Solve the forward kinematics equation to calculate the end pose or solve the inverse kinematics equation to calculate the joint angle; Solving the forward kinematics equations to compute the end pose involves: The end pose is calculated by multiplying the transformation matrices of all joints: T(q)=T1·T2……T n The end position and posture are separated into: Among them, q is the motion angle of the joint, T(q) is the solution of positive kinematics, that is, the end position, p(q) is the end coordinate, and p(q) is the end posture. Solving the inverse kinematics equations to calculate the joint angles includes: Calculate the current end pose error: Position error: Δp=p d -p(q) Attitude error: ΔR=R d ·R(q) T By converting ΔR into a rotation vector Δr, we get: Δr=logΔR Comprehensive error: ΔT=[Δp,Δr] Compute the Jacobian matrix: The Jacobian matrix J(q) represents the partial derivatives of the end pose with respect to the joint angle: Joint variable update: Update the joint angles by solving linear equations: Δq=J(q) -1 ·ΔT Update joint angles: q new =q+Δq Check convergence: If the error ||ΔT||<ε, the iteration is exited, and the joint angle solved by inverse kinematics is q new ; S23: When solving the inverse kinematics equation to calculate the joint angle, the joint sequence number set in the URDF file is used to control the movement of each joint of the spray robot arm to the joint angle solved by the inverse kinematics, thereby realizing the motion control of the robot arm in the simulation system.

3. The simulation method of the spraying robot arm based on the Bullet engine according to claim 2 is characterized in that: In the step S22, in the process of solving the joint motion angle by the inverse kinematics algorithm according to the position information of the target point, it also includes: combining the interpolation algorithm to generate a series of discrete intermediate points between the starting point and the target point, so that the end effector of the robot arm moves along a straight line. The specific steps are as follows: S221: First, interpolate the coordinates of the two points. For linear motion, use the parameter λ∈[0,1] to represent the ratio from the starting point to the end point: p(λ)=(1-λ)·p s +λ·p e Among them, p s is the coordinate information of the starting point, p e is the coordinate information of the end point and the coordinate information of the interpolation point p; According to the number of discretized points N from the starting point to the end point, calculate λ i =i / N, generate position sequence: p i =p(λ i ) S222: Use the quaternion interpolation algorithm for the two-point posture, and convert the rotation matrix R of the starting point and the end point s and R e Convert to quaternion q s and q e , use the spherical linear interpolation algorithm to calculate the quaternion q of the interpolation point i , the quaternion q of the interpolation point i Then convert back to the terminal posture R i ; Among them, the quaternion q i The expression is as follows: Where θ = cos -1 (q s ·q e ) is the angle of the quaternion; S223: For each interpolation point coordinate p i and the terminal posture R i , combined to generate the interpolation point pose P i =[p i ,R i ], and save the result in the interpolation point sequence P; S224: Calculate the joint angle of the robot arm for each interpolation point through the inverse kinematics algorithm, and gradually send joint angle instructions according to the interpolation results to control the movement of the robot arm to achieve linear movement of the robot arm in the simulation system.

4. The simulation method of the spraying robot arm based on the Bullet engine according to claim 1, characterized in that: The step S40 specifically includes the following steps: S41: acquiring point cloud data of each angle of the steel structure by a multi-angle scanner, and obtaining the overall point cloud data of the steel structure by point cloud registration; S42: dividing the obtained overall point cloud data of the steel structure into cross-section data and vertical plate data, and pre-processing the cross-section data and the vertical plate data; S43: Calculate the spraying range of the spraying robot arm in combination with the process parameters of the steel structure, and calculate the number of spraying tracks according to the spraying range of the spraying robot arm; S44: Calculate characteristic points of the steel structure according to the number of spraying trajectories and the spraying range, and convert the characteristic points of the steel structure into the spatial position and spraying posture of the robot arm corresponding to each trajectory; S45: Import the final spraying trajectory of the spraying robot arm into the spraying robot arm model in the simulation environment for verification.

5. The simulation method of the spraying robot arm based on the Bullet engine according to claim 4 is characterized in that: In the step S41, obtaining the overall point cloud data of the steel structure through point cloud registration includes the following steps: S411: Performing rough registration, firstly performing fast point feature histogram feature extraction on the source point cloud and the target point cloud respectively, obtaining the FPFH feature vector of each point, then using the feature matching algorithm to find similar feature vectors in the source point cloud and the target point cloud, and calculating the distance between the two FPFH feature vectors, wherein if the distance is small enough, they are considered to be corresponding point pairs; after finding a set of initial feature matching pairs, using the RANSAC algorithm to estimate the initial transformation matrix, wherein the RANSAC algorithm finds the optimal transformation matrix through random sampling and iterative optimization, and finally obtains the rotation matrix R and translation vector T of rough registration; S412: Perform fine registration, use the initial transformation matrix obtained by coarse registration, and use the ICP algorithm to iteratively optimize to obtain a more accurate solution. The ICP algorithm calculates the distance between the corresponding points of the source point cloud and the target point cloud, constructs the rotation and translation matrix RT, transforms the source point cloud through RT, and calculates the mean square error after the transformation; if the mean square error meets the threshold condition, the algorithm ends, otherwise it continues to iterate until the error meets the threshold condition or the number of iterations is terminated, and finally the overall point cloud data of the steel structure is obtained.

6. The simulation method of a spraying robot arm based on the Bullet engine according to claim 4, characterized in that: In the step S42, dividing the obtained overall point cloud data of the steel structure into cross-section data and vertical plate data specifically includes the following steps: S421: plane cutting the overall point cloud data of the steel structure along its length extension direction in units of K millimeters to obtain multiple three-dimensional cutting sections; S422: According to the type of steel structure, it is divided into two categories: H-shaped steel and square steel. For the cross-section processing of H-shaped steel, the web is used as the symmetry center to divide the cross-section into two independent characteristic structures on the left and right; for the cross-section of square steel, each side of the square cross-section is used as the dividing plane to divide the cross-section into four independent characteristic structures; S423: The characteristic structure divided by the cutting plane is processed as a basic unit, and its envelope data is extracted and described. Finally, the point cloud data of the entire steel structure is further subdivided into cross-section data and facade data.

7. The simulation method of a spraying robot arm based on the Bullet engine according to claim 4, characterized in that: In the step S43, the spraying range of the spraying robot arm is calculated in combination with the process parameters of the steel structure, wherein the spraying range calculation formula is as follows: Among them, S is the spraying range, D is the height of the nozzle from the board surface, and A is the nozzle angle.

8. The simulation method of a spraying robot arm based on the Bullet engine according to claim 6, characterized in that: In step S43, calculating the number of spraying tracks according to the spraying range of the spraying robot arm includes the following steps: S431: Based on the spraying margin in the process parameters and the overlap of the two spraying trajectories, start calculating the first starting point of each set of data; wherein the spraying margin is defined as the increment or decrement of the amount of spraying of the next set after the robot arm completes each set of spraying tasks; the overlap of the two spraying trajectories is deducted from the overall spraying range, and the actual spacing between the trajectories is calculated to further determine the spraying path of the next trajectory; S432: After determining the coordinates of the first point, the remaining spraying range is calculated through the first point and the range of each set of point cloud data; S433: Divide the remaining spraying range by the distance between the two tracks to calculate how many tracks are needed to spray the area; S434: Multiply the number of trajectories by the distance between the trajectories and subtract the remaining spraying range to obtain the remainder of the current set of spraying data, which will be used as a reference for calculating the starting point of the next set of data.

9. The simulation method of a spraying robot arm based on the Bullet engine according to claim 8, characterized in that: In the step S44, the characteristic points of the steel structure are calculated according to the number of spraying trajectories and the spraying range, and the characteristic points of the steel structure are converted into the spatial position of the robot arm and the spraying posture corresponding to each trajectory, which specifically includes the following steps: S441: The first starting point position plus N multiplied by the distance between the two spraying tracks is used to calculate the feature point position of the Nth track; S442: According to the characteristic points of the Nth trajectory, the cross-section data and the vertical plate data within the spraying range of each trajectory are traversed, and the starting point, the ending point, the highest point and the lowest point of each spraying trajectory are determined according to the four key points obtained after preprocessing; S443: After obtaining the trajectory spraying coordinates, they need to be converted into the spatial position (x, y, z) of the robot arm and the corresponding spraying posture (Rx, Ry, Rz); S444: Save the spraying robot arm posture data of each spraying trajectory into a list to form complete spraying trajectory coordinate data.

10. A simulation system for a spraying robot arm based on the Bullet engine, characterized in that: include: A robot arm model creation module is configured to use SolidWorks software in combination with a URDF plug-in to perform three-dimensional modeling based on the spraying robot arm model data, establish basic axes and coordinate systems at the joints of the spraying robot arm, set corresponding kinematic parameters, and export a URDF file of the spraying robot arm; The robot control module is configured to calculate the forward kinematics equations of the motion angles of each joint of the spraying robot arm based on the Bullet engine, or calculate the end position of the spraying robot arm through the motion angles of each joint of the spraying robot arm, and control the spraying robot arm to move to a specified position by using the inverse kinematics equations, so as to realize the motion control of the spraying robot arm in the simulation environment; The work scene import module is configured to build various work scenes of the spraying robot arm based on the Bullet engine and create them in the STL file format, select the corresponding work environment type and add the STL format file of the work environment type to the model of the spraying robot arm; A spraying trajectory calculation module is configured to scan the steel structure and form point cloud data, and generate a spraying trajectory of the spraying robot arm in combination with the spraying process parameters of the steel structure, and import the spraying trajectory into the spraying robot arm model in the simulation environment for verification, wherein if no error occurs during the simulation verification process, the spraying trajectory data is saved; If an error occurs during the simulation, perform corresponding corrections and optimization according to the error type until the simulation is correct.

Images