Simulation method and system of spraying mechanical arm based on bullet engine

Abstract

The application discloses a simulation method and system of a spraying mechanical arm based on a Bullet engine, and the simulation method comprises the following steps: using SolidWorks software combined with a URDF plug-in, performing three-dimensional modeling based on spraying mechanical arm model data, establishing a basic shaft, a coordinate system and setting kinematic parameters at the joint of the spraying mechanical arm, and exporting a URDF file of the spraying mechanical arm; on the basis of the Bullet engine, combining a forward kinematics equation or an inverse kinematics equation to realize motion control of the spraying mechanical arm in a simulation environment; on the basis of the Bullet engine, building multiple working scenes of the spraying mechanical arm and adding them to the model of the spraying mechanical arm; scanning a steel structural part and forming point cloud data, combining spraying process parameters of the steel structural part to generate a spraying track of the spraying mechanical arm, and importing the spraying track into the spraying mechanical arm model in the simulation environment for verification.

Description

Technical Field

[0001] This invention relates to the field of robotic arm spraying control and simulation, and in particular to a simulation method and system for a spraying robotic arm based on the Bullet engine. Background Technology

[0002] In the industrial manufacturing sector, robotic arms are being used more and more extensively, replacing manual labor in various production tasks. However, due to the high cost of robotic arms and the complexity of their operation, direct experimental use would pose unnecessary risks and extremely high costs. Furthermore, the complexity of the real-world environment and the occurrence of unforeseen circumstances necessitate expert judgment to resolve various emergencies. This presents a significant challenge to industrial production.

[0003] In industrial production, steel component painting is an indispensable part of the manufacturing process. It's not just a means of beautifying the steel components' appearance, but also a crucial step in protecting them from corrosion and extending their lifespan. However, in most parts of my country, industrial painting still relies on traditional manual methods. To meet user requirements for paint film thickness and surface quality, additional sanding and touch-up painting are often necessary. This results in high labor intensity, low efficiency, and long-term exposure to paint and solvents that can harm workers' health. Therefore, replacing manual labor with machines to improve production efficiency, quality, and reduce operating costs is becoming the choice of many companies.

[0004] Because robotic arms are expensive, directly conducting steel component painting experiments could lead to collisions between the robotic arm and the steel component, causing damage. Furthermore, the painting experiments would waste painting materials, increasing production costs. Therefore, a robotic arm painting simulation platform is needed to allow workers to simulate and verify painting behavior in a virtual environment, guiding subsequent painting operations and reducing unnecessary losses and waste. However, existing methods for robotic arm painting of steel components mostly design the painting trajectory based on a specific type of steel component. If a new shape of steel component is used, workers need to redesign a painting trajectory, leading to decreased production efficiency. Additionally, designing a robot motion process for a simulation platform is cumbersome, often requiring users to have strong programming skills and system configuration knowledge to define complex motion trajectories and operation sequences. This means that non-professional users may need to spend a significant amount of time learning and adapting to the platform, increasing initial learning costs. Secondly, these platforms may lack pre-defined professional modules and algorithms when dealing with specific fields such as painting processes, forcing users to build from scratch. Summary of the Invention

[0005] This solution addresses the problems and needs raised above by proposing a simulation method and system for a painting robot arm based on the Bullet engine. The above technical objectives can be achieved by adopting the following technical features, and other technical effects are also brought about.

[0006] One object of this invention is to provide a simulation method for a painting robot arm based on the Bullet engine, comprising the following steps:

[0007] S10: Using SolidWorks software in conjunction with the URDF plugin, perform 3D modeling based on the model data of the spraying robot arm, establish basic axes and coordinate systems at the joints of the spraying robot arm and set the corresponding kinematic parameters, and export the URDF file of the spraying robot arm.

[0008] S20: Based on the Bullet engine, it combines the forward kinematic equations that calculate the motion angles of each joint of the spraying robot arm according to the end coordinates and poses of the spraying robot arm, or the inverse kinematic equations that calculate the end pose of the spraying robot arm according to the motion angles of each joint of the spraying robot arm, and control the spraying robot arm to move to the specified position, so as to realize the motion control of the spraying robot arm in the simulation environment.

[0009] S30: Based on the Bullet engine, build various working scenarios for the painting robot arm and create them in STL file format. Select the corresponding working environment type and add the STL file of that working environment type to the model of the painting robot arm.

[0010] S40: Scan the steel structure and generate point cloud data. Combine the spraying process parameters of the steel structure to generate the spraying trajectory of the spraying robot arm. Import the spraying trajectory into the spraying robot arm model in the simulation environment for verification. If no errors occur during the simulation verification, save the spraying trajectory data. If errors occur during the simulation, perform corresponding correction and optimization processing according to the error type until the simulation is error-free.

[0011] Furthermore, the simulation method and system for the painting robotic arm based on the Bullet engine according to the present invention may also have the following technical features:

[0012] In one example of the present invention, step S20 includes the following steps:

[0013] S21: Construct the kinematic model of the robotic arm in the simulation system. The painting robotic arm consists of serially connected joints and links. For an n-degree-of-freedom robotic arm, there are n joints numbered from 1 to n, and n+1 links numbered from 0 to n. Link 0 is the base of the robotic arm, and link n has an end effector. Joint i connects link i and link i-1. Based on the DH parameters, define the transformation matrix T for each joint.i :

[0014]

[0015] Among them, a i Is along X i Axis, from Z i Move to Z i+1 The distance, i.e., the abstract length of link i; θ i It is around Z i Axis, from X i-1 Rotate to X i The angle, i.e., the angle between link i-1 and link i; α i From Z i Rotate to Z i+1 Angle, axis of rotation is X i That is, the angle between link i and link i+1; d i It is along Z i Axis, from X i-1 Move to X i The distance;

[0016] S22: Solve the forward kinematics equations to calculate the end pose or solve the inverse kinematics equations to calculate the joint angles;

[0017] Solving the forward kinematics equations to calculate the end-effector pose includes:

[0018] The end-effector pose is calculated by multiplying the transformation matrices of all joints:

[0019] T(q) = T1·T2……T n

[0020] The end-effector position and attitude are separated as follows:

[0021]

[0022] Where q is the joint motion angle, T(q) is the solution of the forward kinematics, i.e. the end pose, p(q) is the end coordinate, and p(q) is the end attitude.

[0023] Solving the inverse kinematic equations to calculate joint angles includes:

[0024] Calculate the current end-effector pose error:

[0025] Position error:

[0026] Δp=p d -p(q)

[0027] Attitude error:

[0028] ΔR=R d ·R(q) T

[0029] By transforming ΔR into a rotation vector Δr, we obtain:

[0030] Δr=logΔR

[0031] Overall error:

[0032] ΔT=[Δp,Δr]

[0033] Calculate the Jacobian matrix:

[0034] The Jacobian matrix J(q) represents the partial derivative of the end-effector pose with respect to the joint angle:

[0035]

[0036] Joint variable update:

[0037] Update joint angles by solving linear equations:

[0038] Δq=J(q) -1 ·ΔT

[0039] Update joint angles:

[0040] q new =q+Δq

[0041] Check for convergence:

[0042] If the error ||ΔT|| < ε, then exit the iteration. At this point, the joint angle obtained from inverse kinematics is q. new ;

[0043] S23: When solving the inverse kinematics equations to calculate joint angles, the joint numbers set in the URDF file are used to control the movement of each joint of the painting robot arm to the joint angles solved by the inverse kinematics, thereby realizing motion control of the robot arm in the simulation system.

[0044] In one example of the present invention, in step S22, during the process of solving the joint motion angle using an inverse kinematics algorithm based on the pose information of the target point, the method further includes: combining an interpolation algorithm to generate a series of discrete intermediate points between the starting point and the target point, so that the end effector of the robotic arm moves along a straight line. The specific steps are as follows:

[0045] S221: First, interpolate the coordinates of the two points. For linear motion, use the parameter λ∈[0,1] to represent the ratio from the starting point to the ending point.

[0046] p(λ)=(1-λ)·p s +λ·p e

[0047] Where, ps It is the coordinate information of the starting point, p e It contains the coordinates of the endpoint and the coordinates of the p interpolation point;

[0048] Calculate λ based on the number of discretization points N from the starting point to the ending point. i =i / N, generating a position sequence:

[0049] p i =p(λ i )

[0050] S222: Use quaternion interpolation algorithm for two-point attitudes to calculate the rotation matrix R of the start and end points. s and R e Convert to quaternion q s and q e The quaternion q at the interpolation point is calculated using the spherical linear interpolation algorithm. i The quaternion q at the interpolation point i Then switch back to the end attitude R i ; where the quaternion q i The expression is as follows:

[0051]

[0052] Where θ = cos -1 (q s ·q e () is the angle between quaternions;

[0053] S223: For each interpolation point coordinate p i and end attitude R i Combine to generate interpolation point pose P i =[p i ,R i The results are stored in the interpolation point sequence P;

[0054] S224: For each interpolation point, the joint angle of the robotic arm is calculated using an inverse kinematics algorithm. Based on the interpolation results, joint angle commands are sent step by step to control the movement of the robotic arm and realize the linear movement of the robotic arm in the simulation system.

[0055] In one example of the present invention, step S40 specifically includes the following steps:

[0056] S41: The point cloud data of the steel structure component from various angles is obtained by a multi-angle scanner, and the overall point cloud data of the steel structure component is obtained by point cloud registration;

[0057] S42: Divide the obtained overall point cloud data of the steel structure into cross-sectional data and vertical plate data, and preprocess the cross-sectional data and vertical plate data;

[0058] S43: Calculate the spraying range of the spraying robot arm based on the process parameters of the steel structure components, and calculate the number of spraying trajectories based on the spraying range of the spraying robot arm.

[0059] S44: Calculate the feature points of the steel structure based on the number of spraying trajectories and the spraying range, and convert the feature points of the steel structure into the spatial position of the robotic arm and the spraying posture corresponding to each trajectory;

[0060] S45: Import the final spraying trajectory of the spraying robot arm into the spraying robot arm model in the simulation environment for verification.

[0061] In one example of the present invention, obtaining the overall point cloud data of the steel structure component through point cloud registration in step S41 includes the following steps:

[0062] S411: Perform coarse registration. First, perform fast point feature histogram feature extraction on the source point cloud and the target point cloud to obtain the FPFH feature vector of each point. Then, use a feature matching algorithm to find similar feature vectors in the source point cloud and the target point cloud and calculate the distance between two FPFH feature vectors. If the distance is small enough, they are considered to be corresponding point pairs. After finding a set of initial feature matching pairs, use the RANSAC algorithm to estimate the initial transformation matrix. The RANSAC algorithm finds the optimal transformation matrix through random sampling and iterative optimization. Finally, the rotation matrix R and translation vector T of the coarse registration are obtained.

[0063] S412: Perform fine registration. Using the initial transformation matrix obtained from coarse registration, use the ICP algorithm for iterative optimization to obtain a more accurate solution. The ICP algorithm constructs a rotation and translation matrix RT by calculating the distance between corresponding points in the source point cloud and the target point cloud. The source point cloud is transformed by RT, and the mean square error after transformation is calculated. If the mean square error meets the threshold condition, the algorithm ends. Otherwise, it continues to iterate until the error meets the threshold condition or the number of iterations terminates, and finally the overall point cloud data of the steel structure is obtained.

[0064] In one example of the present invention, step S42, dividing the obtained overall point cloud data of the steel structure into cross-sectional data and vertical plate data, specifically includes the following steps:

[0065] S421: Cut the overall point cloud data of the steel structure along its length direction, using K millimeters as the unit, to obtain multiple three-dimensional cutting sections;

[0066] S422: Based on the type of steel structural components, they are divided into two categories: H-beams and U-beams. For the cross-section treatment of H-beams, the cross-section is divided into two independent characteristic structures, with the web as the center of symmetry. For the cross-section of U-beams, each side of the U-shaped cross-section is used as a dividing plane to divide the cross-section into four independent characteristic structures.

[0067] S423: The feature structure divided by the cutting plane is processed as the basic unit, and its envelope data is extracted and described. Finally, the point cloud data of the entire steel structure is further subdivided into cross-sectional data and vertical plate data.

[0068] In one example of the present invention, in step S43, the spraying range of the spraying robot arm is calculated based on the process parameters of the steel structure, wherein the formula for calculating the spraying range is as follows:

[0069]

[0070] Where S is the spraying range, D is the height of the nozzle from the board surface, and A is the nozzle angle.

[0071] In one example of the present invention, in step S43, calculating the number of spraying trajectories based on the spraying range of the spraying robot arm includes the following steps:

[0072] S431: Based on the spraying allowance in the process parameters and the overlap of the two spraying trajectories, start calculating the first starting point of each set of data; where, the spraying allowance is defined as the increment or decrement of the amount of spraying relative to the next set after the robotic arm completes each set of spraying tasks; the overlap of the two spraying trajectories is deducted from the overall spraying range to calculate the actual distance between the trajectories, which is used to further determine the spraying path of the next trajectory.

[0073] S432: After determining the coordinates of the first point, calculate the remaining spraying range using the first point and the range of each set of point cloud data;

[0074] S433: Divide the remaining spraying area by the distance between the two tracks to calculate how many more tracks are needed to spray the area.

[0075] S434: Multiply the number of tracks by the distance between tracks and then subtract the remaining spraying range. The result is the margin of this set of spraying data. This margin will be used as a reference for calculating the starting point of the next set of data.

[0076] In one example of the present invention, in step S44, calculating the feature points of the steel structure based on the number of spraying trajectories and the spraying range, and converting the feature points of the steel structure into the spatial position of the robotic arm and the spraying posture corresponding to each trajectory, specifically includes the following steps:

[0077] S441: The feature point of the Nth trajectory is calculated by adding N to the position of the first starting point and multiplying it by the distance between the two spraying trajectories.

[0078] S442: Based on the feature points of the Nth trajectory, traverse the cross-sectional data and vertical plate data within the spraying range of each trajectory, and determine the start point, end point, highest point and lowest point of each spraying trajectory based on the four key points obtained after preprocessing.

[0079] S443: After obtaining the trajectory spraying coordinates, it is necessary to convert them into the spatial position (x, y, z) of the robotic arm and the corresponding spraying posture (Rx, Ry, Rz);

[0080] S444: Save the pose data of the painting robot arm for each painting trajectory to a list to form complete painting trajectory coordinate data.

[0081] Another object of the present invention is to provide a simulation system for a painting robot arm based on the Bullet engine, comprising:

[0082] The robotic arm model creation module is configured to use SolidWorks software in conjunction with the URDF plugin to perform 3D modeling based on the spraying robotic arm model data, establish basic axes and coordinate systems at the joints of the spraying robotic arm and set the corresponding kinematic parameters, and export the URDF file of the spraying robotic arm.

[0083] The robotic arm control module is configured to, based on the Bullet engine, combine the forward kinematic equations that calculate the motion angles of each joint of the spraying robotic arm according to the coordinates and pose of the end effector of the spraying robotic arm, or the inverse kinematic equations that calculate the end effector pose of the spraying robotic arm according to the motion angles of each joint of the spraying robotic arm, and control the spraying robotic arm to move to a specified position, thereby realizing motion control of the spraying robotic arm in the simulation environment.

[0084] The work scene import module is configured to build various work scenes for the painting robot arm on the Bullet engine and create them in STL file format. Select the corresponding work environment type and add the STL file of that work environment type to the model of the painting robot arm.

[0085] The spraying trajectory calculation module is configured to scan steel structural components and generate point cloud data, and combine the spraying process parameters of the steel structural components to generate the spraying trajectory of the spraying robot arm. The spraying trajectory is then imported into the spraying robot arm model in the simulation environment for verification. If no errors occur during the simulation verification process, the spraying trajectory data is saved; if errors occur during the simulation process, corresponding correction and optimization processes are performed according to the error type until the simulation is error-free.

[0086] Compared with the prior art, the present invention has the following beneficial effects:

[0087] 1. Adaptable to various robotic arm models: This invention uses robotic arm model data and the URDF plugin of SolidWorks software to export all types of robotic arm models into URDF files, enabling them to be simulated and verified in the simulation system.

[0088] 2. Optimize the motion control of the robotic arm: By combining the forward and inverse kinematics algorithms and point-to-point interpolation motion algorithms provided by the Bullet engine, the movement of the robotic arm is made smoother and more natural. In addition, by combining the joint parameter settings in the robotic arm URDF file, the motion logic of the simulated robotic arm is made more similar to the motion process of the real robotic arm, preventing unnecessary risks caused by the difference between the simulation results and the actual output during the actual motion process.

[0089] 3. Automatic generation of robotic arm spraying trajectory: Based on the point cloud data of the steel structure, this invention generates a spraying trajectory adapted to the current steel structure through process parameters and trajectory algorithms. This not only improves the working efficiency of the robotic arm, but also greatly reduces the cost of redesigning the trajectory according to different steel structure requirements.

[0090] 4. Improved Robotic Arm Safety: After the trajectory algorithm calculates the spraying trajectory, the trajectory list is sent to the simulation system for verification. If no collisions or problems occur during the simulation, the trajectory list is then sent to the real robot. Simultaneously, the system automatically generates transition points before the robotic arm begins spraying, ensuring the robot avoids obstacles such as itself, workpieces, and transport vehicles during movement, thus mitigating potential collision risks. This significantly improves the safety of robotic arm operations and reduces the cost and risk of actual operation.

[0091] The preferred embodiments of the invention will be described in more detail below with reference to the accompanying drawings, so as to facilitate an understanding of the features and advantages of the invention. Attached Figure Description

[0092] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments of the present invention will be briefly described below. The drawings are merely illustrative of some embodiments of the present invention and are not intended to limit the scope of the present invention to all embodiments.

[0093] Figure 1 This is a flowchart illustrating the simulation method of a painting robot arm based on the Bullet engine according to an embodiment of the present invention.

[0094] Figure 2 A schematic diagram illustrating the process of constructing a complex motion simulation model of a multi-degree-of-freedom robotic arm according to an embodiment of the present invention;

[0095] Figure 3 This is a schematic diagram illustrating the process of calculating the spraying trajectory from point cloud data of a steel structural component according to an embodiment of the present invention. Detailed Implementation

[0096] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The same reference numerals in the drawings represent the same components. It should be noted that the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0097] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms “first,” “second,” and similar terms used in this patent application specification and claims do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, “an” or “a” and similar terms do not necessarily indicate a quantity limitation. Terms such as “comprising” or “including” mean that the element or object preceding the word encompasses the element or object listed following the word and its equivalents, without excluding other elements or objects. Terms such as “connected” or “linked” are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as “upper,” “lower,” “left,” and “right” are used only to indicate relative positional relationships; these relative positional relationships may change accordingly when the absolute position of the described object changes.

[0098] According to a first aspect of the present invention, a simulation method for a painting robotic arm based on a Bullet engine is provided, such as... Figure 1 As shown, it includes the following steps:

[0099] S10: As Figure 2 As shown, using SolidWorks software with the URDF plugin, a 3D model of the spraying robot arm is created based on the model data. Basic axes, coordinate systems, and corresponding kinematic parameters are established at the joints of the spraying robot arm, and the URDF file of the spraying robot arm is exported.

[0100] Specifically, to improve the efficiency and safety of robotic arms in real-world work scenarios, acquiring and constructing a motion simulation model of the robotic arm is crucial. By combining a 3D model of an object in the work environment with a realistic simulation environment, the working conditions of the robotic arm during actual operation can be simulated more comprehensively. Using this simulation environment, potential problems can be identified in advance, and operational procedures can be optimized, thereby reducing potential risks and losses in actual operations. In this way, not only can work efficiency be improved, but also the malfunctions and safety hazards caused by incorrect operation or improper design during robotic arm operation can be significantly reduced.

[0101] like Figure 2 As shown, firstly, the corresponding 3D model file is obtained based on the selected robotic arm model. Then, these 3D model files are imported into SolidWorks software for further conversion into URDF files, which are used for control and motion simulation in the simulation system. URDF files are an XML-based format specifically designed to describe the geometric and motion information of a robot structure. For a robotic arm, the URDF file needs to clearly describe the relative relationships between each link, joint types, and their parameters, thus providing a foundation for subsequent calls to Bullet engine control functions to achieve precise control of the robotic arm's motion.

[0102] Robotic arms typically consist of several links and joints between them. Therefore, when generating a URDF file in SolidWorks, reference geometric points must first be inserted based on the geometric characteristics of each link. These points will serve as the origin reference positions of the coordinate system. The joint type between each link also needs to be defined according to its motion mode. Common joint types include rotary joints and sliding joints. For rotary joints, a reference axis needs to be inserted into the model as the joint's rotation axis. Based on the inserted coordinate points and the reference axis, a coordinate system is established for each link. To ensure that the robotic arm can be displayed and move correctly in the simulation system, special attention needs to be paid to the direction of the coordinate axes when establishing the coordinate system for each link. The Z-axis of the coordinate system of the robotic arm base must be consistent with the vertical direction of the robotic arm; otherwise, the robotic arm may appear upside down in the simulation system. In addition, the Z-axis of the coordinate system of all links should be parallel to the rotation axis of the corresponding joint to ensure that the motion in the simulation conforms to the actual motion mode of the robotic arm.

[0103] After establishing the rotation axis and coordinate system, we can then launch the URDF plugin to configure the robot arm's parameters. First, configure the robot arm's base. In this step, we need to name the base links and set their coordinate system to the previously created base coordinate system. In the link components, select the part representing the base in the model and set the number of sub-links to 1. Next, continue configuring the base's sub-link parameters. When configuring the sub-links, in addition to setting the same parameters as the base links, we also need to name the joints between the base and the sub-links. The joint's motion type and the reference coordinate axis for joint motion also need to be configured accordingly. Then, based on the number of links and joints in the robot arm, gradually set the parameters for each link and its corresponding joint. Once all link and joint parameters are set, enter the URDF file export interface. When exporting the URDF file, depending on the different robot arm models, further set the maximum and minimum rotation angles, torque, and rotation speeds for each joint. After completing all parameter settings, the robot arm's URDF file can be exported.

[0104] Furthermore, it's crucial to note that when importing URDF files into the simulation system, the position of each link in the robotic arm is determined by its center coordinates. This mechanism causes the end effector of the generated robotic arm model in the simulation system to align with the center coordinates of the last link, rather than the actual end effector. Simultaneously, the overall position coordinates of the robotic arm will be based on the center coordinates of the base, not the position coordinates of the lowest point of the robotic arm. Therefore, to ensure more accurate position and end effector coordinates in the simulation system, we need to add two virtual links to the generated URDF file. Specifically, the child link of the first virtual link should be set to the base, and its joint motion type should be fixed. The parent link of the second virtual link should be the last link of the robotic arm, also with a fixed joint motion type. These two virtual links do not require parameters such as collision detection or material appearance settings. This approach ensures more accurate position and end effector coordinates of the robotic arm in the simulation environment, thereby optimizing the simulation results.

[0105] By following the steps above, the 3D model of the robotic arm is accurately converted into a URDF file. This not only lays the foundation for the control of the robotic arm in the simulation system, but also provides more accurate parameter support for subsequent calls to the control functions of the simulation engine, ensuring that the motion trajectory and operational stability during the simulation process are realistically simulated.

[0106] S20: Based on the Bullet engine, it combines the forward kinematic equations that calculate the motion angles of each joint of the spraying robot arm according to the end coordinates and poses of the spraying robot arm, or the inverse kinematic equations that calculate the end pose of the spraying robot arm according to the motion angles of each joint of the spraying robot arm, and control the spraying robot arm to move to the specified position, so as to realize the motion control of the spraying robot arm in the simulation environment.

[0107] S30: Based on the Bullet engine, various working scenarios for the painting robot arm are built and created in STL file format. The corresponding working environment type is selected, and the STL file of that type is added to the model of the painting robot arm. Specifically, this invention designs an STL file import module based on the Bullet engine, which can efficiently create the required working scenarios using STL file format. This process involves not only the geometry of the scene but also the coordinate information of the objects' specific positions within the scene. This method makes building complex working scenarios in the simulation system more intuitive and convenient. After the scene is built, it can be saved as a JSON file for easy import into the simulation system later. This method allows for quick loading of previously built scenes, saving time and effort. Furthermore, the simulation system also supports directly importing pre-built scene files, providing greater flexibility. Whether reusing previous scenes or quickly building working environments in new projects, the STL file import module can efficiently and flexibly build and manage simulation working scenarios.

[0108] S40: Scan the steel structure and generate point cloud data. Combine the spraying process parameters of the steel structure to generate the spraying trajectory of the spraying robot arm. Import the spraying trajectory into the spraying robot arm model in the simulation environment for verification. If no errors occur during the simulation verification, save the spraying trajectory data. If errors occur during the simulation, perform corresponding correction and optimization processing according to the error type until the simulation is error-free.

[0109] This simulation method is adaptable to various robotic arm models. By using robotic arm model data and the URDF plugin of SolidWorks software, this invention can export all types of robotic arm models into URDF files, enabling them to be simulated and verified in the simulation system.

[0110] This simulation method can optimize the motion control of the robotic arm: by combining the forward and inverse kinematics algorithms and point-to-point interpolation motion algorithms provided by the Bullet engine, the movement of the robotic arm is made smoother and more natural. Furthermore, by combining the joint parameter settings in the robotic arm's URDF file, the motion logic of the simulated robotic arm is made more similar to the motion process of the real robotic arm, preventing unnecessary risks caused by the difference between the simulation results and the actual output during the real motion process.

[0111] This simulation method can automatically generate the spraying trajectory of the robotic arm: Based on the point cloud data of the steel structure, this invention generates a spraying trajectory adapted to the current steel structure through process parameters and trajectory algorithms, thereby not only improving the working efficiency of the robotic arm, but also greatly reducing the cost of redesigning the trajectory according to different steel structure requirements.

[0112] This simulation method improves the safety of robotic arms: after the trajectory algorithm calculates the spraying trajectory, the trajectory list is sent to the simulation system for verification. In the simulation system, if no collisions or problems occur during trajectory execution, the trajectory list is then sent to the real robot. Simultaneously, the system automatically generates transition points before the robotic arm begins spraying, ensuring that the robot avoids obstacles such as itself, workpieces, and transport vehicles during movement, thus avoiding potential collision risks. This significantly improves the safety of robotic arm operations and reduces the cost and risk of actual operation.

[0113] In one example of the present invention, such as Figure 2 As shown, step S10 includes the following:

[0114] By using SolidWorks software and the URDF plugin, users can efficiently model robotic arm files from various manufacturers. The first step is to establish basic axes and coordinate systems for each joint of the robotic arm. This step is fundamental to ensuring the correct movement of each joint. Next, parameters such as joint rotation angles, speeds, and torques need to be set. These settings not only affect the robotic arm's motion performance but also its efficiency and safety in actual operation. Precise parameter configuration ensures that the robotic arm exhibits motion characteristics consistent with actual operation in the simulation environment.

[0115] After completing the above steps, export the URDF (Unified Robot Description Format) file for the robotic arm. Generating this file is a crucial step in the entire modeling process, providing essential data support for subsequent simulation experiments. Finally, import the generated URDF file into the simulation platform to conduct various simulation experiments. By combining SolidWorks software with the URDF plugin, users can systematically build robotic arm models, completing the entire process from modeling to simulation experiments, laying a solid foundation for the practical application of robotic arms.

[0116] In one example of the present invention, step S20 includes the following steps:

[0117] S21: Construct the kinematic model of the robotic arm in the simulation system. The painting robotic arm consists of serially connected joints and links. For an n-degree-of-freedom robotic arm, there are n joints numbered from 1 to n, and n+1 links numbered from 0 to n. Link 0 is the base of the robotic arm, which is generally fixed. Link n has an end effector. Joint i connects link i and link i-1. Based on the DH (Denavit-Hartenberg) parameters, define the transformation matrix T for each joint. i :

[0118]

[0119] Among them, a i Is along X i Axis, from Z i Move to Z i+1 The distance, i.e., the abstract length of link i; θ i It is around Z i Axis, from X i-1 Rotate to X i The angle, i.e., the angle between link i-1 and link i; α i From Z i Rotate to Z i+1 Angle, axis of rotation is X i That is, the angle between link i and link i+1; d i It is along Z i Axis, from X i-1 Move to X i The distance;

[0120] S22: Solve the forward kinematics equations to calculate the end pose or solve the inverse kinematics equations to calculate the joint angles;

[0121] Solving the forward kinematics equations to calculate the end-effector pose includes:

[0122] The end-effector pose is calculated by multiplying the transformation matrices of all joints:

[0123] T(q) = T1·T2……T n

[0124] The end-effector position and attitude are separated as follows:

[0125]

[0126] Where q is the joint motion angle, T(q) is the solution of the forward kinematics, i.e. the end pose, p(q) is the end coordinate, and p(q) is the end attitude.

[0127] Solving the inverse kinematic equations to calculate joint angles includes:

[0128] Calculate the current end-effector pose error:

[0129] Position error:

[0130] Δp=p d -p(q)

[0131] Attitude error:

[0132] ΔR=R d ·R(q) T

[0133] By transforming ΔR into a rotation vector Δr, we obtain:

[0134] Δr=logΔR

[0135] Overall error:

[0136] ΔT=[Δp,Δr]

[0137] Calculate the Jacobian matrix:

[0138] The Jacobian matrix J(q) represents the partial derivative of the end-effector pose with respect to the joint angle:

[0139]

[0140] Joint variable update:

[0141] Update joint angles by solving linear equations:

[0142] Δq=J(q) -1 ·ΔT

[0143] Update joint angles:

[0144] q new =q+Δq

[0145] Check for convergence:

[0146] If the error ||ΔT|| < ε, then exit the iteration. At this point, the joint angle obtained from inverse kinematics is q. new ;

[0147] S23: When solving the inverse kinematics equations to calculate joint angles, the joint numbers set in the URDF file are used to control the movement of each joint of the painting robot arm to the joint angles solved by the inverse kinematics, thereby realizing motion control of the robot arm in the simulation system.

[0148] In one example of the present invention, in step S22, during the process of solving the joint motion angle using an inverse kinematics algorithm based on the pose information of the target point, the method further includes: combining an interpolation algorithm to generate a series of discrete intermediate points between the starting point and the target point, so that the end effector of the robotic arm moves along a straight line. The specific steps are as follows:

[0149] S221: First, interpolate the coordinates of the two points. For linear motion, use the parameter λ∈[0,1] to represent the ratio from the starting point to the ending point.

[0150] p(λ)=(1-λ)·p s +λ·p e

[0151] Where, p s It is the coordinate information of the starting point, p e It contains the coordinates of the endpoint and the coordinates of the p interpolation point;

[0152] Calculate λ based on the number of discretization points N from the starting point to the ending point. i =i / N, generating a position sequence:

[0153] p i =p(λ i )

[0154] S222: Use quaternion interpolation algorithm for two-point attitudes to calculate the rotation matrix R of the start and end points. s and R e Convert to quaternion q s and q e The quaternion q at the interpolation point is calculated using the spherical linear interpolation algorithm. i The quaternion q at the interpolation point i Then switch back to the end attitude R i ; where the quaternion q i The expression is as follows:

[0155]

[0156] Where θ = cos -1 (q s ·q e () is the angle between quaternions;

[0157] S223: For each interpolation point coordinate P i and end attitude R i Combine to generate interpolation point pose P i =[P i ,R i The results are stored in the interpolation point sequence P;

[0158] S224: For each interpolation point, the joint angle of the robotic arm is calculated using an inverse kinematics algorithm. Based on the interpolation results, joint angle commands are sent step by step to control the movement of the robotic arm and realize the linear movement of the robotic arm in the simulation system.

[0159] In the simulation system, the joint motion angle is solved by inverse kinematics algorithm based on the pose information of the target point. However, with only the target pose information, the movement process of the robotic arm to the target point may not be a linear motion, which will lead to the inability to control the motion trajectory. Therefore, this invention combines an interpolation algorithm to generate a series of discrete intermediate points between the starting point and the target point, so that the end effector of the robotic arm moves in a straight line.

[0160] First, the pose information of the robotic arm in the obtained trajectory list is used to solve the motion angles of each joint of the robotic arm using inverse kinematics algorithms based on the Bullet engine. Then, based on the Bullet engine and the joint parameters configured in the robotic arm's URDF file, each joint motor of the robotic arm is controlled to move to the joint angle solved by inverse kinematics according to the sequence number set for each joint in the URDF file. Since the joint motion angles are solved by inverse kinematics based on the pose information of two points in the simulation environment, but only the joint angles of two points are available, the movement of the robotic arm between the two points may not be linear. Therefore, an interpolation method is needed to subdivide the distance between the two points into multiple two-point movements. By moving gradually, the robotic arm can smoothly complete the movement from one point to another on the simulation platform, avoiding abrupt changes that could cause problems with the simulated trajectory path.

[0161] In one example of the present invention, such as Figure 3 As shown, step S40 specifically includes the following steps:

[0162] S41: The point cloud data of the steel structure component from various angles is obtained by a multi-angle scanner, and the overall point cloud data of the steel structure component is obtained by point cloud registration;

[0163] S42: Divide the obtained overall point cloud data of the steel structure into cross-sectional data and vertical plate data, and preprocess the cross-sectional data and vertical plate data;

[0164] S43: Calculate the spraying range of the spraying robot arm based on the process parameters of the steel structure components, and calculate the number of spraying trajectories based on the spraying range of the spraying robot arm.

[0165] S44: Calculate the feature points of the steel structure based on the number of spraying trajectories and the spraying range, and convert the feature points of the steel structure into the spatial position of the robotic arm and the spraying posture corresponding to each trajectory;

[0166] S45: Import the final spraying trajectory of the spraying robot arm into the spraying robot arm model in the simulation environment for verification.

[0167] Specifically, a multi-angle scanner is used to perform a detailed scan of the steel structure to be coated, generating corresponding 3D data files. First, multiple scanning devices are set up at different angles of the steel structure to comprehensively cover all parts of the structure. During the scanning process, these devices generate multiple sets of point cloud files, each corresponding to the scan results at different angles. Subsequently, a point cloud registration algorithm is used to register the point cloud data from different angles into a complete point cloud model of the steel structure. This step ensures that the geometry of the steel structure can be completely and accurately reproduced. After obtaining the complete point cloud data, the simulation system further applies specialized cross-section and vertical plate processing algorithms to extract cross-section and vertical plate data. When the simulation system receives the cross-section and vertical plate data generated by the vision algorithm, it first preprocesses this data. The main goal of preprocessing is to extract precise coordinate information that can be used for further calculations. This coordinate information is combined with set coating parameters, such as the height between the nozzle and the plate surface, the angle between the nozzle and the plate surface, and the nozzle coverage. Combining these parameters, the system calculates the optimal coating trajectory using a trajectory algorithm. The trajectory includes the spatial position, movement path, and spraying posture of the nozzle during the spraying process. The generated spraying trajectory will be verified on a simulation system to ensure its rationality and accuracy. After successful simulation verification, the system will save the final generated spraying trajectory to a trajectory list. This trajectory list will be used for subsequent actual spraying operations as the basis for controlling the robotic arm to perform the spraying task.

[0168] In one example of the present invention, obtaining the overall point cloud data of the steel structure component through point cloud registration in step S41 includes the following steps:

[0169] S411: Perform coarse registration. First, perform fast point feature histogram feature extraction on the source point cloud and the target point cloud to obtain the FPFH feature vector of each point. Then, use a feature matching algorithm to find similar feature vectors in the source point cloud and the target point cloud and calculate the distance between two FPFH feature vectors. If the distance is small enough, they are considered to be corresponding point pairs. After finding a set of initial feature matching pairs, use the RANSAC algorithm to estimate the initial transformation matrix. The RANSAC algorithm finds the optimal transformation matrix through random sampling and iterative optimization. Finally, the rotation matrix R and translation vector T of the coarse registration are obtained.

[0170] S412: Perform fine registration. Using the initial transformation matrix obtained from coarse registration, use the ICP algorithm for iterative optimization to obtain a more accurate solution. The ICP algorithm constructs a rotation and translation matrix RT by calculating the distance between corresponding points in the source point cloud and the target point cloud. The source point cloud is transformed by RT, and the mean square error after transformation is calculated. If the mean square error meets the threshold condition, the algorithm ends. Otherwise, it continues to iterate until the error meets the threshold condition or the number of iterations terminates, and finally the overall point cloud data of the steel structure is obtained.

[0171] Specifically, the process of obtaining the robotic arm's motion trajectory using trajectory algorithms by combining steel structure point cloud data is as follows: Figure 3 As shown, multi-angle point cloud data is acquired using a multi-angle scanner. Point cloud registration is then performed on the multi-angle point cloud files in two steps. The first step is coarse registration, which is performed when the source and target point clouds have no known relative initial positions. Generally, a method based on local feature description is used for coarse registration. First, Fast Point Feature Histogram (FPFH) feature extraction is performed on both the source and target point clouds to obtain the FPFH feature vector for each point, which describes the local geometry of that point. Then, a feature matching algorithm is used to find similar feature vectors in the source and target point clouds. A common metric is Euclidean distance, which calculates the distance between two FPFH feature vectors. If the distance is small enough, they are considered corresponding point pairs. In the feature matching process, Euclidean distance is used to measure the similarity between two feature vectors. After finding an initial set of feature matching pairs, the RANSAC (Random Sample Consensus) algorithm is used to estimate the initial transformation matrix. The RANSAC algorithm finds the optimal transformation matrix through random sampling and iterative optimization. Finally, the rotation matrix R and translation vector T of the coarse registration are obtained. The second step is fine registration, which is a more precise and refined registration based on the coarse registration. Using the initial transformation matrix obtained from the coarse registration, the ICP (Iterative Closest Point) algorithm is used for iterative optimization to obtain a more accurate solution. The ICP algorithm constructs a rotation and translation matrix RT by calculating the distance between corresponding points in the source and target point clouds. The source point cloud is then transformed using RT, and the mean square error (MSE) after the transformation is calculated. If the MSE meets a threshold condition, the algorithm terminates. Otherwise, it continues iterating until the error meets the threshold condition or the iteration count terminates. Through these two stages of registration calculations (coarse and fine registration), the overall point cloud data of the steel structure can be obtained.

[0172] In one example of the present invention, step S42, dividing the obtained overall point cloud data of the steel structure into cross-sectional data and vertical plate data, specifically includes the following steps:

[0173] S421: Cut the overall point cloud data of the steel structure along its length direction, using K millimeters as the unit, to obtain multiple three-dimensional cutting sections;

[0174] S422: Based on the type of steel structural components, they are divided into two categories: H-beams and U-beams. For the cross-section treatment of H-beams, the cross-section is divided into two independent characteristic structures, with the web as the center of symmetry. For the cross-section of U-beams, each side of the U-shaped cross-section is used as a dividing plane to divide the cross-section into four independent characteristic structures.

[0175] S423: The feature structure divided by the cutting plane is processed as the basic unit, and its envelope data is extracted and described. Finally, the point cloud data of the entire steel structure is further subdivided into cross-sectional data and vertical plate data.

[0176] Specifically, the obtained overall point cloud data of the steel structure is divided into cross-sectional data and vertical plate data. First, the overall point cloud data of the steel structure is cut along its length direction, with each cut in millimeters (K), resulting in multiple three-dimensional cross-sections. Based on the type of steel structure, it can be divided into H-beams and U-beams. For the cross-section processing of H-beams, the web is used as the center of symmetry, dividing the cross-section into two independent feature structures (left and right). For the cross-section of U-beams, each side of the U-shaped cross-section is used as a dividing plane, dividing the cross-section into four independent feature structures. Each cutting plane is processed based on the aforementioned "unidirectional feature block" as the basic unit, extracting and describing its envelope data. Through this cutting process, the point cloud data of the steel structure can be refined, allowing various feature structures to be expressed independently in three-dimensional space. Finally, the point cloud data of the entire steel structure is further subdivided into cross-sectional data and vertical plate data. The obtained cross-sectional and vertical plate data packages all contain clear start and end markers to ensure data integrity and identifiability. Based on the identified steel structure type, these data can be divided into: H-beams and U-beams. Each type of steel structure corresponds to a sequential number for easy tracking and referencing in data management and analysis. The data is primarily divided into cross-sectional data and vertical plate data, with each data type having a corresponding number of data points. Cross-sectional data supplements the common data shared by both types. Each cross-sectional data point includes the following additional information to identify its sequential number, the X-axis coordinate value, the number of normal vectors and envelope points, the coordinates of the center point, and unidirectional feature module data: this part varies depending on whether it's H-beams or U-beams, with 2 or 4 unidirectional feature modules per module. Each module is numbered 0, 1, 2 and 0, 1, 2, 3. In addition, the unidirectional feature module data includes: the normal vector coordinates of the first unidirectional feature module and the coordinates of the envelope points for each unidirectional feature module. Vertical plate data, in addition to the cross-sectional data, includes the number of height points and the specific height values ​​corresponding to the vertical plate data type.

[0177] In one example of the present invention, the preprocessing of the cross-sectional data and vertical plate data in step S42 specifically includes the following steps:

[0178] The point cloud coordinates of the steel structure are transformed from the original reference coordinate system to the world coordinate system of the simulation system through affine transformation.

[0179] The point cloud data of the steel structure is divided into N groups of data according to the working range of the robotic arm, and the length of each group of data is the working range of the robotic arm.

[0180] For individual cross-sectional data and vertical plate data, find the maximum and minimum values ​​in the Z and Y directions. These four points will serve as the starting and ending points for subsequent trajectory planning, providing necessary reference information for trajectory calculation.

[0181] The preprocessed cross-sectional data and vertical plate data are packaged into a TXT file for subsequent processing.

[0182] First, the cross-sectional and vertical plate data of the steel structure are preprocessed to extract coordinate information for subsequent trajectory calculations. Since the point cloud coordinates of the steel structure components are generated based on the reference coordinate system during scanning, a coordinate system transformation is required to achieve more accurate positioning in the simulation system. Specifically, an affine transformation is used to convert the point cloud coordinates of the steel structure from the original reference coordinate system to the world coordinate system of the simulation system, ensuring a unified spatial reference in the simulation environment. After the coordinate system transformation, the point cloud data of the steel structure is divided into N groups according to the working range of the robotic arm, with each group's length corresponding to the robotic arm's working range. For individual cross-sectional and vertical plate data, the maximum and minimum values ​​in the Z and Y axes are found; these four points will serve as the starting and ending points for subsequent trajectory planning, providing necessary reference information for trajectory calculations. Finally, the preprocessed cross-sectional and vertical plate data are packaged into a TXT file for subsequent processing.

[0183] In one example of the present invention, in step S43, the spraying range of the spraying robot arm is calculated based on the process parameters of the steel structure, wherein the formula for calculating the spraying range is as follows:

[0184]

[0185] Where S is the spraying range, D is the height of the nozzle from the board surface, and A is the nozzle angle.

[0186] In one example of the present invention, in step S43, calculating the number of spraying trajectories based on the spraying range of the spraying robot arm includes the following steps:

[0187] S431: Based on the spraying allowance in the process parameters and the overlap of the two spraying trajectories, start calculating the first starting point of each set of data; where, the spraying allowance is defined as the increase or decrease in the amount of spraying relative to the next set after the robotic arm completes each set of spraying tasks, which means that the robotic arm may spray more or less between different sets; the overlap of the two spraying trajectories is deducted from the overall spraying range to calculate the actual distance between the trajectories, which is used to further determine the spraying path of the next trajectory.

[0188] S432: After determining the coordinates of the first point, calculate the remaining spraying range using the first point and the range of each set of point cloud data;

[0189] S433: Divide the remaining spraying area by the distance between the two tracks to calculate how many more tracks are needed to spray the area.

[0190] S434: Multiply the number of tracks by the distance between tracks and then subtract the remaining spraying range. The result is the margin of this set of spraying data. This margin will be used as a reference for calculating the starting point of the next set of data.

[0191] In one example of the present invention, in step S44, calculating the feature points of the steel structure based on the number of spraying trajectories and the spraying range, and converting the feature points of the steel structure into the spatial position of the robotic arm and the spraying posture corresponding to each trajectory, specifically includes the following steps:

[0192] S441: The feature point of the Nth trajectory is calculated by adding N to the position of the first starting point and multiplying it by the distance between the two spraying trajectories.

[0193] S442: Based on the feature points of the Nth trajectory, traverse the cross-sectional data and vertical plate data within the spraying range of each trajectory, and determine the start point, end point, highest point and lowest point of each spraying trajectory based on the four key points obtained after preprocessing.

[0194] S443: After obtaining the trajectory spraying coordinates, it is necessary to convert them into the spatial position (x, y, z) of the robotic arm and the corresponding spraying posture (Rx, Ry, Rz);

[0195] S444: Save the pose data of the painting robot arm for each painting trajectory to a list to form complete painting trajectory coordinate data.

[0196] In one example of the present invention, step S45, importing the final spraying trajectory of the spraying robot arm into the spraying robot arm model in the simulation environment for verification, includes:

[0197] The system selects between optimizing the trajectory or directly outputting a trajectory list based on the simulation results. If no errors occur during simulation, the system saves the trajectory list for later use on the actual robot for painting. However, if errors occur during simulation, the system handles them accordingly based on the error type. For example, if the error is due to exceeding the external axis's range of motion or a collision, the trajectory is deemed unusable, and the system recalculates a new painting trajectory and performs simulation verification again. On the other hand, if the simulation verification shows no exceedance of the external axis's range of motion but a collision occurs, the system further checks the distance between the nozzle and the panel. If this distance is within the set range, the system adjusts the external axis distance and recalculates to generate a new painting trajectory. Conversely, if the distance between the nozzle and the panel is found to exceed the set range, the system modifies the distance and then recalculates to generate a new painting trajectory. This process repeats until the painting trajectory is error-free in the simulation verification.

[0198] In one example of the present invention, after step S20, the method further includes: based on the Bullet engine, using a physics-based collision detection algorithm, adding a bounding box to the painting robot model to determine whether the generated painting trajectory will cause a collision between the robot and the object.

[0199] This invention, based on the Bullet engine, utilizes a physics-based collision detection algorithm to add bounding boxes to object models. This process aims to simulate interactions and collisions between objects, thereby achieving a more realistic physical simulation. The simulation system performs collision detection on the bounding box of each object. By detecting the bounding box, the system can determine whether the generated spraying trajectory will cause a collision between the robotic arm and the object. This determination not only helps identify potential collision risks but also provides a basis for subsequent path optimization. Based on the collision detection results, the generated spraying path can be optimized to ensure that the realistic robotic arm can effectively avoid obstacles such as itself, workpieces, and transport vehicles during movement. This optimization aims to reduce the probability of collisions and enhance the safety and efficiency of operation.

[0200] Another object of the present invention is to provide a simulation system for a painting robot arm based on the Bullet engine, comprising:

[0201] The robotic arm model creation module is configured to use SolidWorks software in conjunction with the URDF plugin to perform 3D modeling based on the spraying robotic arm model data, establish basic axes and coordinate systems at the joints of the spraying robotic arm and set the corresponding kinematic parameters, and export the URDF file of the spraying robotic arm.

[0202] The robotic arm control module is configured to, based on the Bullet engine, combine the forward kinematic equations (calculated from the end-effector coordinates and pose) to determine the motion angles of each joint of the painting robotic arm, or the inverse kinematic equations (calculated from the joint motion angles) to determine the end-effector pose and control the robotic arm to a specified position. This enables motion control of the painting robotic arm within a simulation environment. Specifically, it controls the movement of the robotic arm. Users can select the desired robot within the scene and operate different robots separately. For the selected robot's external axis drive, raising or lowering the robotic arm body avoids movement to singularities within the robotic arm's working area, optimizing the trajectory path. It acquires and displays robotic arm joint data, obtaining the end-effector position information based on different reference coordinate systems (world coordinate system or robotic arm coordinate system) or controlling the robotic arm's movement by setting the end-effector position. Users can also set the end-effector position coordinates or joint angle parameters to control the robotic arm to a specified position.

[0203] The work scene import module is configured to build various work scenes for the painting robot arm based on the Bullet engine and create them in STL file format. It allows users to select the appropriate work environment type and add its STL file to the painting robot arm model. Specifically, it allows browsing the model file directory and replacing parts of the model according to the specific work scene. The module can export and save the already built work scene as a configuration file, or select a configuration file to import. After successfully importing the work scene, the system allows replacement of some entity models within the scene, facilitating dynamic adjustments to the model according to actual needs. After replacement, refreshing the scene generates the updated work environment, displaying the replaced model. This module not only improves the flexibility of model management but also makes model configuration and loading in different work scenes more efficient. Users can select entity models in the work scene and perform model operations, such as reading the current position, moving the model, adding collision volumes, and removing collision volumes. After loading the model, the system automatically reads the current position of the selected model and allows users to select the model and manipulate its spatial coordinates (x, y, z) and Euler angles (r) of its orientation. x ,r y ,r z The parameter is displayed in the parameter list. After resetting and adjusting the current position of the model as needed, you can manually adjust the current position of the model by clicking "Move Model".

[0204] The spraying trajectory calculation module is configured to scan steel structural components and generate point cloud data. Combined with the spraying process parameters of the steel structural components, it generates the spraying trajectory for a spraying robot arm. This trajectory is then imported into a simulation environment's spraying robot arm model for verification. If no errors occur during simulation verification, the spraying trajectory data is saved. If errors occur during simulation, appropriate corrections and optimizations are performed based on the error type until the simulation is error-free. Specifically, it generates the spraying trajectory for the robot arm's movement from the steel structure's point cloud data and the trajectory algorithm. The scanned steel structure data is preprocessed and packaged into a TXT file, saved locally. Then, based on either the vertical plate data or the cross-sectional data, a motion trajectory is generated using the trajectory algorithm, and the motion is simulated in the simulation scene. To facilitate a visual view of the trajectory path in the simulation scene, trajectory tracking is added to the robot arm using the Bullet engine, leaving running traces at the robot arm's end effector. After the robot arm moves correctly in the scene, the trajectory list is saved and displayed in the trajectory list window. Furthermore, after modifying the process parameters or the robot arm's external axis, clicking the trajectory optimization button allows for recalculation and verification of the trajectory.

[0205] This simulation system can adapt to various robotic arm models. By using robotic arm model data and the URDF plugin of SolidWorks software, this invention can export all types of robotic arm models into URDF files, which can then be used for simulation verification in the simulation system.

[0206] This simulation system can optimize the motion control of the robotic arm: by combining the forward and inverse kinematics algorithms and point-to-point interpolation motion algorithms provided by the Bullet engine, the movement of the robotic arm is made smoother and more natural. Furthermore, by combining the joint parameter settings in the robotic arm's URDF file, the motion logic of the simulated robotic arm is made more similar to the motion process of the real robotic arm, preventing unnecessary risks caused by the difference between the simulation results and the actual output during the real motion process.

[0207] This simulation system can automatically generate the spraying trajectory of the robotic arm: Based on the point cloud data of the steel structure, this invention generates a spraying trajectory adapted to the current steel structure through process parameters and trajectory algorithms, thereby not only improving the working efficiency of the robotic arm, but also greatly reducing the cost of redesigning the trajectory according to different steel structure requirements.

[0208] This simulation system enhances the safety of robotic arms: after the trajectory algorithm calculates the spraying trajectory, it sends a list of trajectories to the simulation system for verification. If no collisions or problems occur during the simulation, the trajectory list is then sent to the real robot. Simultaneously, the system automatically generates transition points before the robotic arm begins spraying, ensuring the robot avoids obstacles such as itself, workpieces, and transport vehicles during its movement, thus mitigating potential collision risks. This significantly improves the safety of robotic arm operations and reduces the cost and risk of actual operation.

[0209] In one example of the invention, it further includes a process module configured to input process parameters, including nozzle height from the board, spray trajectory overlap, nozzle-board angle, and nozzle angle. The process parameters can be saved and loaded, and the margin obtained from the trajectory calculation can be displayed simultaneously.

[0210] In one example of the invention, a view control module is also included, configured to display the constructed simulation scene. By unlocking the view state, a suitable angle can be selected to observe the robotic arm's painting process. Alternatively, while the view is locked, a point selection function can be enabled to obtain the position of the selected point in the world coordinate system. If the robotic arm collides with other rigid objects during its movement, a collision point will be left at the collision location and displayed on the interface for trajectory optimization and safety checks.

[0211] The foregoing description, with reference to preferred embodiments, details the exemplary implementation of the simulation method and system for the painting robot based on the Bullet engine proposed in this invention. However, those skilled in the art will understand that various modifications and alterations can be made to the above specific embodiments without departing from the concept of this invention, and various combinations can be made to the various technical features and structures proposed in this invention without exceeding the protection scope of this invention, which is determined by the appended claims.

Claims

1. A simulation method for a painting robot arm based on the Bullet engine, characterized in that, Includes the following steps: S10: Using SolidWorks software with the URDF plugin, perform 3D modeling based on the spraying robot model data, establish basic axes and coordinate systems at the joints of the spraying robot and set the corresponding kinematic parameters, and export the URDF file of the spraying robot. S20: Based on the Bullet engine, it combines the forward kinematic equations that calculate the motion angles of each joint of the spraying robot arm according to the end coordinates and poses of the spraying robot arm, or the inverse kinematic equations that calculate the end pose of the spraying robot arm according to the motion angles of each joint of the spraying robot arm, and control the spraying robot arm to move to the specified position, so as to realize the motion control of the spraying robot arm in the simulation environment. S30: Based on the Bullet engine, build various working scenarios for the painting robot arm and create them in STL file format. Select the corresponding working environment type and add the STL file of that working environment type to the model of the painting robot arm. S40: Scan the steel structure and form point cloud data, and generate the spraying trajectory of the spraying robot arm by combining the spraying process parameters of the steel structure. Import the spraying trajectory into the spraying robot arm model in the simulation environment for verification. If no errors occur during the simulation verification process, save the spraying trajectory data. If errors occur during the simulation, perform corresponding correction and optimization processes according to the error type until the simulation is error-free.

2. The simulation method for the painting robot arm based on the Bullet engine according to claim 1, characterized in that, Step S20 includes the following steps: S21: Construct the kinematic model of the robotic arm in the simulation system. The painting robotic arm consists of serially connected joints and links. For an n-degree-of-freedom robotic arm, there are n joints numbered from 1 to n, and n+1 links numbered from 0 to n. Link 0 is the base of the robotic arm, and link n has an end effector. Joint i connects link i and link i-1. Based on the DH parameters, define the transformation matrix T for each joint. i : Among them, a i Is along X i Axis, from Z i Move to Z i+1 The distance, i.e., the abstract length of link i; θ i It is around Z i Axis, from X i-1 Rotate to X i The angle, i.e., the angle between link i-1 and link i; α i From Z i Rotate to Z i+1 Angle, axis of rotation is X i That is, the angle between link i and link i+1; d i It is along Z i Axis, from X i-1 Move to X i The distance; S22: Solve the forward kinematics equations to calculate the end pose or solve the inverse kinematics equations to calculate the joint angles; Solving the forward kinematics equations to calculate the end-effector pose includes: The end-effector pose is calculated by multiplying the transformation matrices of all joints: T(q)=T1·T2……T n The end-effector position and attitude are separated as follows: Where q is the joint motion angle, T(q) is the solution of the forward kinematics, i.e., the end-effector pose, p(q) is the end-effector coordinate, and p(q) is the end-effector attitude. Solving the inverse kinematic equations to calculate joint angles includes: Calculate the current end-effector pose error: Position error: Δp=p d -p(q) Attitude error: ΔR=R d ·R(q) T By transforming ΔR into a rotation vector Δr, we obtain: Δr=logΔR Overall error: ΔT=[Δp,Δr] Calculate the Jacobian matrix: The Jacobian matrix J(q) represents the partial derivative of the end-effector pose with respect to the joint angle: Joint variable update: Update joint angles by solving linear equations: Δq=J(q) -1 ·ΔT Update joint angles: q new =q+Δq Check for convergence: If the error ||ΔT|| < ε, then exit the iteration. At this point, the joint angle obtained from inverse kinematics is q. new ; S23: When solving the inverse kinematics equations to calculate joint angles, the joint numbers set in the URDF file are used to control the movement of each joint of the painting robot arm to the joint angles solved by the inverse kinematics, thereby realizing motion control of the robot arm in the simulation system.

3. The simulation method for the painting robot arm based on the Bullet engine according to claim 2, characterized in that, In step S22, during the process of solving the joint motion angle using the inverse kinematics algorithm based on the pose information of the target point, the method further includes: combining an interpolation algorithm to generate a series of discrete intermediate points between the starting point and the target point, so that the end effector of the robotic arm moves along a straight line. The specific steps are as follows: S221: First, interpolate the coordinates of the two points. For linear motion, use the parameter λ∈[0,1] to represent the ratio from the starting point to the ending point. p(λ)=(1-λ)·p s +λ·p e Where, p s It is the coordinate information of the starting point, p e It contains the coordinates of the endpoint and the coordinates of the p interpolation point; Calculate λ based on the number of discretization points N from the starting point to the ending point. i =i / N, generating a position sequence: p i =p(λ i ) S222: Use quaternion interpolation algorithm for two-point attitudes to calculate the rotation matrix R of the start and end points. s and R e Convert to quaternion q s and q e The quaternion q at the interpolation point is calculated using the spherical linear interpolation algorithm. i The quaternion q at the interpolation point i Then switch back to the end attitude R i ; where the quaternion q i The expression is as follows: Where θ = cos -1 (q s ·q e () is the angle between quaternions; S223: For each interpolation point coordinate p i and end attitude R i Combine to generate interpolation point pose P i =[p i ,R i The results are stored in the interpolation point sequence P; S224: For each interpolation point, the joint angle of the robotic arm is calculated using an inverse kinematics algorithm. Based on the interpolation results, joint angle commands are sent step by step to control the movement of the robotic arm and realize the linear movement of the robotic arm in the simulation system.

4. The simulation method for the painting robot arm based on the Bullet engine according to claim 1, characterized in that, Step S40 specifically includes the following steps: S41: The point cloud data of the steel structure component from various angles is obtained by a multi-angle scanner, and the overall point cloud data of the steel structure component is obtained by point cloud registration; S42: Divide the obtained overall point cloud data of the steel structure into cross-sectional data and vertical plate data, and preprocess the cross-sectional data and vertical plate data; S43: Calculate the spraying range of the spraying robot arm based on the process parameters of the steel structure components, and calculate the number of spraying trajectories based on the spraying range of the spraying robot arm. S44: Calculate the feature points of the steel structure based on the number of spraying trajectories and the spraying range, and convert the feature points of the steel structure into the spatial position of the robotic arm and the spraying posture corresponding to each trajectory; S45: Import the final spraying trajectory of the spraying robot arm into the spraying robot arm model in the simulation environment for verification.

5. The simulation method for a painting robot based on the Bullet engine according to claim 4, characterized in that, In step S41, obtaining the overall point cloud data of the steel structure component through point cloud registration includes the following steps: S411: Perform coarse registration. First, perform fast point feature histogram feature extraction on the source point cloud and the target point cloud to obtain the FPFH feature vector of each point. Then, use a feature matching algorithm to find similar feature vectors in the source point cloud and the target point cloud and calculate the distance between two FPFH feature vectors. If the distance is small enough, they are considered to be corresponding point pairs. After finding a set of initial feature matching pairs, use the RANSAC algorithm to estimate the initial transformation matrix. The RANSAC algorithm finds the optimal transformation matrix through random sampling and iterative optimization. Finally, the rotation matrix R and translation vector T of the coarse registration are obtained. S412: Perform fine registration. Using the initial transformation matrix obtained from coarse registration, use the ICP algorithm for iterative optimization to obtain a more accurate solution. The ICP algorithm constructs a rotation and translation matrix RT by calculating the distance between corresponding points in the source point cloud and the target point cloud. The source point cloud is transformed by RT, and the mean square error after transformation is calculated. If the mean square error meets the threshold condition, the algorithm ends. Otherwise, it continues to iterate until the error meets the threshold condition or the number of iterations terminates, and finally the overall point cloud data of the steel structure is obtained.

6. The simulation method for a painting robot based on the Bullet engine according to claim 4, characterized in that, In step S42, dividing the obtained overall point cloud data of the steel structure into cross-sectional data and vertical plate data specifically includes the following steps: S421: Cut the overall point cloud data of the steel structure along its length direction, using K millimeters as the unit, to obtain multiple three-dimensional cutting sections; S422: Based on the type of steel structural components, they are divided into two categories: H-beams and U-beams. For the cross-section treatment of H-beams, the cross-section is divided into two independent characteristic structures, with the web as the center of symmetry. For the cross-section of U-beams, each side of the U-shaped cross-section is used as a dividing plane to divide the cross-section into four independent characteristic structures. S423: The feature structure divided by the cutting plane is processed as the basic unit, and its envelope data is extracted and described. Finally, the point cloud data of the entire steel structure is further subdivided into cross-sectional data and vertical plate data.

7. The simulation method for a painting robot based on the Bullet engine according to claim 4, characterized in that, In step S43, the spraying range of the spraying robot arm is calculated based on the process parameters of the steel structure components. The formula for calculating the spraying range is as follows: Where S is the spraying range, D is the height of the nozzle from the board surface, and A is the nozzle angle.

8. The simulation method for a painting robot based on the Bullet engine according to claim 6, characterized in that, In step S43, calculating the number of spraying trajectories based on the spraying range of the spraying robot arm includes the following steps: S431: Based on the spraying allowance in the process parameters and the overlap of the two spraying trajectories, start calculating the first starting point of each set of data; where, the spraying allowance is defined as the increment or decrement of the amount of spraying relative to the next set after the robotic arm completes each set of spraying tasks; the overlap of the two spraying trajectories is deducted from the overall spraying range to calculate the actual distance between the trajectories, which is used to further determine the spraying path of the next trajectory. S432: After determining the coordinates of the first point, calculate the remaining spraying range using the first point and the range of each set of point cloud data; S433: Divide the remaining spraying area by the distance between the two tracks to calculate how many more tracks are needed to spray the area. S434: Multiply the number of tracks by the distance between tracks and then subtract the remaining spraying range. The result is the margin of this set of spraying data. This margin will be used as a reference for calculating the starting point of the next set of data.

9. The simulation method for a painting robot based on the Bullet engine according to claim 8, characterized in that, In step S44, the characteristic points of the steel structure are calculated based on the number of spraying trajectories and the spraying range, and the characteristic points of the steel structure are converted into the spatial position of the robotic arm and the spraying posture corresponding to each trajectory. This specifically includes the following steps: S441: The feature point of the Nth trajectory is calculated by adding N to the position of the first starting point and multiplying it by the distance between the two spraying trajectories. S442: Based on the feature points of the Nth trajectory, traverse the cross-sectional data and vertical plate data within the spraying range of each trajectory, and determine the start point, end point, highest point and lowest point of each spraying trajectory based on the four key points obtained after preprocessing. S443: After obtaining the trajectory spraying coordinates, it is necessary to convert them into the spatial position (x, y, z) of the robotic arm and the corresponding spraying posture (Rx, Ry, Rz); S444: Save the pose data of the painting robot arm for each painting trajectory to a list to form complete painting trajectory coordinate data.

10. A simulation system for a painting robotic arm based on the Bullet engine, characterized in that, include: The robotic arm model creation module is configured to use SolidWorks software in conjunction with the URDF plugin to perform 3D modeling based on the spraying robotic arm model data, establish basic axes and coordinate systems at the joints of the spraying robotic arm and set the corresponding kinematic parameters, and export the URDF file of the spraying robotic arm. The robotic arm control module is configured to, based on the Bullet engine, combine the forward kinematic equations that calculate the motion angles of each joint of the spraying robotic arm according to the coordinates and pose of the end effector of the spraying robotic arm, or the inverse kinematic equations that calculate the end effector pose of the spraying robotic arm according to the motion angles of each joint of the spraying robotic arm, and control the spraying robotic arm to move to a specified position, thereby realizing motion control of the spraying robotic arm in the simulation environment. The work scene import module is configured to build various work scenes for the painting robot arm on the Bullet engine and create them in STL file format. Select the corresponding work environment type and add the STL file of that work environment type to the model of the painting robot arm. The spraying trajectory calculation module is configured to scan the steel structure and form point cloud data, and generate the spraying trajectory of the spraying robot arm in combination with the spraying process parameters of the steel structure. The spraying trajectory is then imported into the spraying robot arm model in the simulation environment for verification. If no errors occur during the simulation verification process, the spraying trajectory data is saved. If errors occur during the simulation, perform corresponding correction and optimization processes according to the error type until the simulation is error-free.

Images