Steel structure scattered workpiece spraying operation planning method based on virtual simulation

Abstract

The invention relates to the technical field of spraying control, in particular to a steel structure scattered workpiece spraying operation planning method based on virtual simulation. The method comprises the following steps: modeling and assembling a physical spraying station to obtain a virtual spraying station; establishing a real-time communication channel between the virtual spraying station and the physical spraying station, and carrying out data binding; respectively acquiring point cloud data of the physical product and the virtual spraying product model, and recording the point cloud data as first point cloud data and second point cloud data; obtaining a final rotation translation matrix by using the first point cloud data and the second point cloud data; executing the final rotation translation matrix on the pre-planned spraying track sequence to obtain a spraying track sequence; a physical spraying station is controlled through the spraying track sequence and IO signals, and product spraying is conducted; and during spraying, the real-time communication channel is used for obtaining data in the mechanical arm controller to optimize the spraying track. According to the invention, the purposes of efficient planning and real-time control of the spraying operation of the steel structure scattered workpieces can be achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of spraying control, and in particular to a method for planning spraying operations of scattered steel workpieces based on virtual simulation. Background Art

[0002] With the development of industrial robots and automation, the spraying method for scattered steel structure workpieces has gradually evolved from manual spraying to robotic automated spraying, eliminating the need for workers to wear protective clothing and hold a spray gun in the harsh spraying environment. There are currently two main types of automated spraying operations for spray robots. One is manual teaching, where the robot motion parameters for the spray path are manually set, and the robot then sprays according to the preset motion parameters. The other is to add equipment such as visual cameras and utilize methods such as pattern matching to achieve autonomous robot path planning.

[0003] Scattered steel workpieces are characterized by a wide variety of types and structures. Conveyors are typically used for loading, making it difficult to deliver workpieces to the spraying station in a fixed position. Therefore, the spraying method must adapt to changes in workpiece position and plan an efficient spraying path. Furthermore, for production lines that produce a wide variety of small-product categories, robots require different trajectories for different positions and products. Therefore, the coordinate relationship between the robot, workpiece, and equipment must be fully considered to ensure the robot's trajectory is compatible.

[0004] However, for the autonomous robot path planning method, many optimization problems need to be solved for different industries, different products, and different application scenarios. The existing spraying path planning is mostly for a single product. When changing to another product, the environment needs to be arranged to determine the spraying path, and the simulation cannot be updated in real time as the spraying progresses, resulting in inaccurate spraying. Summary of the Invention

[0005] In order to solve the above technical problems, the purpose of the present invention is to provide a method for planning spraying operations for scattered steel workpieces based on virtual simulation. The technical solutions adopted are as follows:

[0006] One embodiment of the present invention provides a method for planning spraying operations for scattered steel workpieces based on virtual simulation, the method comprising: 1. A method for planning spraying operations for scattered steel workpieces based on virtual simulation, characterized in that the method comprises:

[0007] Modeling the spraying robot arm, nozzle tool, peripheral equipment and spraying product respectively to obtain a virtual spraying robot arm model, a virtual nozzle tool model, a virtual peripheral equipment model and a virtual spraying product model; assembling the virtual spraying robot arm model, the virtual nozzle tool model, the virtual peripheral equipment model and the virtual spraying product model to obtain a virtual spraying station;

[0008] Establish a real-time communication channel between the virtual spraying station and the physical spraying station, and bind the virtual spraying station with the physical spraying station;

[0009] Setting a pre-planned spraying trajectory sequence; obtaining point cloud data of a physical product and a virtual spraying product model, respectively, recorded as first point cloud data and second point cloud data; obtaining a final rotation and translation matrix using the first point cloud data and the second point cloud data; executing the final rotation and translation matrix on the pre-planned spraying trajectory sequence to obtain a spraying trajectory sequence;

[0010] The spraying trajectory sequence and IO signals are used to control the physical spraying station to spray the product. During spraying, the real-time communication channel is used to obtain data from the robotic arm controller to optimize the spraying trajectory.

[0011] Preferably, obtaining a virtual spraying robot arm model includes:

[0012] Obtaining a virtual spraying robot arm model includes motion joint hierarchical modeling, motion joint coordinate system origin modeling, and zero-point posture calibration.

[0013] Preferably, the virtual printhead tool model includes:

[0014] Perform three-dimensional visual modeling on the nozzle tool, and bind the nozzle tool obtained by the three-dimensional visual modeling to the end of the virtual spraying robot arm model; bind the nozzle process parameters to the nozzle tool obtained by the three-dimensional visual modeling to obtain a virtual nozzle tool model.

[0015] Preferably, the virtual spraying robot arm model, the virtual spray head tool model, the virtual peripheral equipment model and the virtual spraying product model are assembled to obtain a virtual spraying station, including:

[0016] According to the CAD layout file, the virtual spraying robot arm model, virtual nozzle tool model, virtual peripheral equipment model and virtual spraying product model are assembled to obtain a virtual spraying station, wherein the assembly constraint methods include fixed constraint, coaxial constraint, angle constraint and contact constraint.

[0017] Preferably, establishing a real-time communication channel between the virtual spraying station and the physical spraying station includes:

[0018] Establishing a real-time communication channel between the virtual spraying station and the physical spraying station includes establishing real-time communication between the virtual spraying station and the mechanical arm of the physical spraying station, and establishing real-time communication between the virtual spraying station and the physical spraying station master executable logic controller.

[0019] Preferably, obtaining a final rotation and translation matrix using the first point cloud data and the second point cloud data includes:

[0020] The initial rotation and translation matrix is ​​obtained by performing rough alignment on the first point cloud data and the second point cloud data; based on the ICP algorithm, the initial rotation and translation matrix is ​​performed on the second point cloud data, and the upper limit of the number of iterations based on the ICP algorithm and the error threshold parameter are set; for each point in the first point cloud data, the nearest point is found in the second point cloud data as the corresponding point; the error measure between the transformed second point cloud data and the first point cloud data is calculated using the mean square error; the optimal rotation matrix and translation matrix are solved based on the corresponding point pairs using the least squares method to minimize the error; the obtained rotation and translation matrix is ​​applied to the point cloud to be aligned, and its position is updated to make it closer to the reference point cloud; the iterative optimization is repeated, and after each iteration, it is checked whether the error is less than the set error threshold parameter. If so, the final rotation and translation matrix is ​​calculated.

[0021] Preferably, performing coarse registration on the first point cloud data and the second point cloud data to obtain an initial rotation and translation matrix includes:

[0022] Based on the RANSAC search method, the threshold delt is set, a matching point set is constructed, 3 pairs of matching points are selected and put into the matching point set, the rotation and translation transformation matrix between the matching points is calculated, and the rotation and translation transformation matrix is ​​applied to the second point cloud data to calculate the Euclidean distance between the two point clouds. If it is less than the threshold delt, the matching point set is updated, otherwise, matching points are continued to be selected to supplement the matching points, and the number of iterations N is set to obtain the initial rotation and translation matrix.

[0023] Preferably, the spraying trajectory sequence and IO signal are used to control the physical spraying station to spray the product, including:

[0024] Construct JSON format data of spray trajectory sequence and IO signal; construct ABB robot control statements based on JSON format data, and use ABB robot control statements to control the physical spray station and perform product spraying.

[0025] Preferably, the method of using a real-time communication channel to obtain data from a robotic arm controller to optimize the spraying trajectory includes:

[0026] The end position coordinates N of the real-time joint angles of the robotic arm at the first N trajectory points are calculated using the robotic arm forward kinematics. x , N y , Nz , alt, belt, gama;

[0027] The Euclidean distance calculation method is used to calculate the absolute error between the real-time end position coordinates of the N robotic arms and the N trajectory points of the spray trajectory sequence;

[0028] The absolute error is thresholded to determine whether error optimization is required. If optimization is required, the average deviation calculation method is used to calculate the deviation x', y', z', alt', belt', gama';

[0029] The deviation of N trajectory positions is calibrated, and the inverse kinematics is used to calculate the inverse kinematics of the calibrated N trajectory points. The calibrated data values ​​are output to the robot controller to optimize the spraying trajectory.

[0030] The embodiments of the present invention have at least the following beneficial effects: the present invention obtains a virtual spraying station by modeling and assembling a spraying robot arm, a nozzle tool, peripheral equipment and a spraying product respectively, thereby providing an accurate simulation environment for spraying simulation and a method for quickly building a virtual simulation environment for spraying. For scenarios such as new equipment and new robot arms in an enterprise, on-site operators can complete the construction of the virtual simulation environment, thereby reducing manpower maintenance costs and improving process switching efficiency; furthermore, a real-time communication channel between the virtual spraying station and the physical spraying station is established, and the virtual spraying station is connected to the physical spraying station. The data of the spraying stations are bound to facilitate data exchange between the virtual spraying station and the physical spraying station; further, the final rotation and translation matrix is ​​executed on the pre-planned spraying trajectory sequence to obtain the spraying trajectory sequence; the spraying trajectory sequence and IO signal are used to control the physical spraying station to spray the product, which meets the mixed-line production spraying of scattered workpieces of steel structures. There is no need for manual teaching of robot spraying, no line stoppage, and the spraying trajectory planning can be automatically performed; when spraying, the real-time communication channel is used to obtain data in the robot arm controller to optimize the spraying trajectory to ensure the spraying quality and spraying efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 A flowchart of a method for planning spraying operations for scattered steel workpieces based on virtual simulation provided by an embodiment of the present invention;

[0033] Figure 2A schematic diagram of a mechanical arm structure for a method for planning spraying operations for scattered steel workpieces based on virtual simulation provided by an embodiment of the present invention;

[0034] Figure 3 A schematic diagram of a cylindrical rotation and translation structure of a method for planning spraying operations for scattered steel workpieces based on virtual simulation provided by an embodiment of the present invention;

[0035] Figure 4 A schematic diagram of the joint structure of a method for planning spraying operations for scattered steel workpieces based on virtual simulation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] To further illustrate the technical means and effects employed by the present invention to achieve the intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail a method for planning spraying operations for scattered steel workpieces based on virtual simulation, including its specific implementation, structure, features, and effects. In the following description, references to different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0037] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0038] The following describes in detail a specific scheme of a method for planning spraying operations for scattered steel workpieces based on virtual simulation provided by the present invention in conjunction with the accompanying drawings.

[0039] Example:

[0040] The main application scenarios of the present invention are as follows: The present invention addresses the limitations of spray robot trajectory planning during the spraying process of scattered steel structure workpieces, which makes it difficult to cope with the changing requirements of different workpieces and different postures, and cannot quickly solve the robot trajectory mapping. It is also impossible to remotely monitor the robot spraying process, and the spraying process consumes a high manpower cost, which affects the spraying processing efficiency. A method for planning spraying operations for scattered steel structure workpieces based on virtual simulation is provided. By using virtual physical model modeling and CAD file mapping methods, a virtual three-dimensional spraying production line is quickly constructed, and real-time communication between physical spraying equipment and virtual spraying equipment is quickly established. Visual recognition technology is used to match the workpiece posture, automatically plan the spraying operation trajectory, and output robot code programs that are suitable for different brands of robotic arms, thereby improving the company's production efficiency and reducing production costs.

[0041] See also Figure 1 , which shows a flow chart of a method for planning spraying operations for scattered steel workpieces based on virtual simulation according to an embodiment of the present invention. The method includes the following steps:

[0042] Step S1, modeling the spraying robot arm, nozzle tool, peripheral equipment and spraying product respectively to obtain a virtual spraying robot arm model, a virtual nozzle tool model, a virtual peripheral equipment model and a virtual spraying product model; assembling the virtual spraying robot arm model, the virtual nozzle tool model, the virtual peripheral equipment model and the virtual spraying product model to obtain a virtual spraying station.

[0043] Through virtual physical model mapping, a virtual three-dimensional spraying production line is quickly constructed. Real-time communication is quickly established between physical spraying equipment and virtual spraying equipment. Through real-time planning steps of spraying trajectory, visual matching technology is used to match workpiece postures, automatically plan spraying operation trajectories, output robot code programs that are suitable for different brands of robotic arms, and control the spraying process in real time, thereby improving enterprise production efficiency and reducing production costs.

[0044] First of all, when simulating spraying, it is necessary to build an accurate virtual simulation environment; therefore, it is necessary to perform virtual modeling on the physical spraying station, specifically the physical spraying robot arm, nozzle tool, peripheral equipment and spraying products.

[0045] Modeling of the spraying robot arm: 3D visualization 1:1 mapping modeling of the spraying robot arm at the spraying station, and modeling of the joint motion of the robot arm based on the motion principle of the robot arm. Specifically including:

[0046] Step 1: Hierarchical modeling of motion joints. The motion forms of the spraying station robot arm are usually divided into rectangular coordinate translation type, cylindrical rotation and translation mixed type, and joint rotation type.

[0047] The rectangular coordinate translation type consists of three mutually orthogonal moving pairs in the arm, which drive the wrist to move linearly along the directions of the three coordinate axes X, Y, and Z. This type of motion hierarchical modeling, such as Figure 2 The robotic arm structure requires the model to be split into four parts. The model hierarchy is as follows Figure 2 ,A, B, C, and D in the figure represent the four split components;

[0048] The arm of the cylindrical rotation-translation hybrid type consists of a revolute joint and two translation joints. This type of motion level modeling, such as Figure 3 The model needs to be split into four parts, and the model hierarchy is as follows Figure 3 As shown, E, F, G, and H in the figure represent four split parts;

[0049] Joint rotation type, each joint is composed of a dynamic rotation joint, and the relative angular displacement of the adjacent parts of the arm is used as the motion coordinate. This type of motion hierarchical modeling, such as Figure 4The model needs to be split into four parts, and the model level interface is as follows Figure 4 As shown, I, J, K, and L in the figure represent four split components.

[0050] Step 2: Model the origin of the motion joint coordinate system. The motion operations of the 3D virtual model based on WebGL are based on the world coordinate system or the local coordinate system. Rotation and translation matrices are applied to control the motion of the virtual robotic arm. The origin of the local coordinate system is crucial for virtual motion operations.

[0051] For the method of determining the origin of the local coordinate system of the revolute pair, determine the rotation axis and the extension line of the connecting rod, and the intersection is the origin of the motion joint coordinate system.

[0052] For the method of determining the origin of the local coordinate system of the moving pair, the direction of the connecting rod is determined to be the moving axis, and the origin of the local coordinate system can be on the moving axis.

[0053] Step 3: Zero-point pose calibration. Each kinematic joint, translation joint, and revolute joint of the physical robotic arm has a zero-point pose, meaning each joint is at zero angle or zero displacement. The zero-point pose of the virtual 3D model is adjusted based on the zero-point pose of the physical robotic arm. This creates a virtual spraying robotic arm model.

[0054] Nozzle tool modeling: 3D visualization modeling of the nozzle tool at the end of the robotic arm, and modeling of spraying process parameters based on the spraying principle of the nozzle. Specifically including:

[0055] The three-dimensional visualization modeling and binding of the nozzle tool is carried out according to the spraying tool of the spraying production line. At the same time, the nozzle tool obtained by the three-dimensional visualization modeling is model-bound to the end of the virtual spraying robot arm model. The hierarchical relationship is the sub-level of the end of the virtual spraying robot arm model.

[0056] Furthermore, the spraying process parameters of the nozzle tool are modeled. These process parameters directly impact the quality of the virtual simulation spraying. These parameters include nozzle diameter L (in mm), spray width F (in mm), and paint spray volume C (in mL / min). These process parameters are bound to the nozzle tool generated through 3D visualization to create a virtual nozzle tool model.

[0057] Modeling of equipment around the spraying station. The equipment around the spraying station mainly includes other equipment models except the core moving parts. In order to perform interference verification of the spraying trajectory and optimize the path, supporting 3D modeling is required, including:

[0058] 3D visualization modeling, 1:1 mapping modeling of the operating table of the spraying production line, the conveyor line of scattered steel structure workpieces, etc.

[0059] Motion modeling of steel structure scattered workpiece conveyor lines. Motion modeling of steel structure scattered workpiece conveyor lines based on motion principles. The conveying principle of the conveyor line is to rely on gravity and friction to transport scattered steel structure workpieces. Abstract modeling of the motion principle, configuration of the friction coefficient SF of the conveyor line material c , elastic coefficient SM e , thus the virtual peripheral device model can be obtained.

[0060] Spraying product modeling, build a 3D model of the spraying product (steel structure scattered workpiece products), and perform 3D visualization 1:1 mapping modeling. At the same time, configure the material friction coefficient GF for the workpiece material friction coefficient c , elastic coefficient GM e .

[0061] In this way, a virtual spraying robot arm model, a virtual spraying tool model, a virtual peripheral equipment model and a virtual spraying product model can be obtained.

[0062] Furthermore, it is necessary to perform virtual mapping on the assembly of the physical spraying station, perform virtual entity assembly mapping based on the CAD layout file, and assemble the virtual spraying robot arm model, virtual nozzle tool model, virtual peripheral equipment model and virtual spraying product model to obtain a complete automated virtual spraying station. The equipment assembly constraint methods include fixed constraints, coaxial constraints, angle constraints and contact constraints, specifically including:

[0063] Fixed constraint assembly,according to the CAD layout file, the virtual model is fixed in the 3D scene,with fixed world coordinates.

[0064] Coaxial constraint assembly: according to the CAD layout file, the axes of the two virtual 3D models coincide and are bound.

[0065] Angle constraint assembly: According to the CAD layout file, there is a specified angle between the two virtual model planes, and the angle is used for model binding assembly.

[0066] Contact constraint assembly: According to the CAD layout file, a contact relationship is established between two parts to perform model binding assembly.

[0067] At this point, a three-dimensional environment for virtual simulation is obtained, and an accurate virtual mapping of the entity's physical spraying station is performed to obtain a virtual spraying station.

[0068] Step S2: establishing a real-time communication channel between the virtual spraying station and the physical spraying station, and binding data between the virtual spraying station and the physical spraying station.

[0069] Furthermore, real-time communication is required between the virtual spraying station and the physical spraying station to establish a real-time communication channel to facilitate data transmission and control of spraying. The key to the actual application of virtual simulation spraying control in actual scenarios lies in the establishment of real-time communication with physical entity equipment.

[0070] Establishing a real-time communication channel between the virtual spraying station and the physical spraying station includes establishing real-time communication between the virtual spraying station and the mechanical arm of the physical spraying station, and establishing real-time communication between the virtual spraying station and the physical spraying station master executable logic controller.

[0071] For the robotic arm control system, build real-time information, collect robotic arm motion data, and establish robot execution code communication logic; the robotic arm motion data is as follows:

[0072] Table 1

[0073]

[0074] Establish communication logic between the virtual simulation and the robot's executable code. The virtual simulation method relies on the robot's executable code to control the robotic arm. The virtual simulation outputs code to the robotic arm controller interface, which then executes the spray trajectory logic code. The robot's executable code communication logic utilizes the UDP communication protocol, with communication messages using the GBK string binary encoding format to improve the robot controller's code parsing speed.

[0075] Establish real-time communication between the virtual spraying station and the physical spraying station master executable logic controller, connect to the master executable logic controller, use ModbusTCP, OPCUA and other standard protocols, the point information is as follows Table 2:

[0076] Table 2

[0077]

[0078]

[0079] Furthermore, after establishing a real-time communication channel between the virtual spraying station and the physical spraying station, it is necessary to bind the virtual spraying station to the physical spraying station, that is, to bind the established real-time communication to the virtual 3D model data;

[0080] Specifically, the real-time data of the robot arm in the physical spraying station is bound to the virtual spraying robot arm model, the real-time joint coordinate data of the robot arm is bound to the rotation pair, and the real-time displacement coordinates of the robot arm are bound to the moving pair.

[0081] The physical spray station master station executes real-time data from the logic controller and binds it to the virtual spray station. Spray start and stop signals are bound to the virtual printhead tool model, enabling virtual spraying effects based on these signals. Spraying process parameters are bound to the virtual spray station, allowing for real-time simulation of spraying effects, including spray flow and spray range, based on these process parameters.

[0082] Step S3, set a pre-planned spray trajectory sequence; obtain point cloud data of the physical product and the virtual spray product model respectively, recorded as the first point cloud data and the second point cloud data; use the first point cloud data and the second point cloud data to obtain the final rotation and translation matrix; execute the final rotation and translation matrix on the pre-planned spray trajectory sequence to obtain the spray trajectory sequence.

[0083] Spray trajectory pre-planning: pre-drawing of spray trajectory based on the constructed 3D model of scattered steel structure workpieces. Commonly used drawing trajectories include zigzag paths and zigzag trajectories. At the same time, for some void areas, the process parameters of the nozzle are fully considered, and the nozzle tool process parameters are set and matched during the trajectory pre-drawing process, thereby setting a pre-planned spray trajectory sequence.

[0084] Spraying 3D model matching. As one of the links in the product production line, the spraying station needs to obtain the type of product currently being sprayed for small-batch, multi-variety production lines. The production line needs to have production line management. After the previous process of the spraying station is completed, it needs to report the work operation to the production management system and report the type of product completed. The spraying virtual simulation system connects to the production management system through an interface protocol to obtain the product type, and matches the 3D model of the product to be sprayed in the 3D model library of scattered steel structure workpieces, so as to then calculate the workpiece posture and optimize the spraying trajectory interference.

[0085] Workpiece pose calculation. Scattered steel workpieces are characterized by a wide variety of types and structures. They are usually loaded using conveyor lines. It is difficult for the workpieces to be transported to the spraying station in a fixed posture. Therefore, after matching the three-dimensional spraying model, the workpiece pose calculation is also required.

[0086] A 3D camera is used to collect 3D point cloud data of the physical product (the scattered steel structure workpiece to be sprayed), i.e., the 3D point cloud of the workpiece to be sprayed, recorded as the first point cloud data. A 3D point set is constructed for the vertices of the virtual spray product model, i.e., the virtual model point cloud, recorded as the second point cloud data.

[0087] The workstation posture calculation adopts the point cloud registration method to calculate the rotation matrix between the second point cloud data and the first point cloud data. The process is divided into two stages: coarse registration and fine registration. The first is the coarse registration stage. The present invention selects the RANSAC search method, and the threshold delt is user-defined in the invention, which can be customized according to the size of scattered steel structure workpieces. Initially construct a matching point set, select 3 pairs of matching points and put them into the matching point set, calculate the rotation and translation transformation matrix between the matching points, apply the rotation and translation transformation matrix to the second point cloud data to calculate the Euclidean distance between the two point clouds, and if it is less than the threshold delt, update the matching point set, otherwise continue to select matching points to supplement the matching points, set the number of iterations N, and thus obtain the initial rotation and translation matrix.

[0088] Furthermore, a fine registration step is performed. The fine registration step is to further accurately calculate the rotation and translation matrix based on the initial rotation and translation matrix of the rough registration in the previous step. The present invention adopts the ICP algorithm for calculation. First, the second point cloud data is subjected to an initial rotation and translation matrix, and the upper limit of the number of iterations of the algorithm and the error threshold parameter are set. For each point in the first point cloud data, the nearest point in the second point cloud data is found as the corresponding point. The mean square error is used to calculate the error metric between the second point cloud data after transformation and the first point cloud data. The least squares method is used to solve the optimal rotation matrix and translation matrix based on the corresponding point pairs to minimize the error metric. The obtained rotation and translation matrix is ​​applied to the point cloud to be registered, and its position is updated to make it close to the reference point cloud. Repeat the iterative optimization. After each iteration, check whether the error change is less than the preset threshold. If so, it is considered that the algorithm has converged and the iteration can be terminated to calculate the final rotation and translation matrix.

[0089] Furthermore, the final rotation and translation matrix is ​​executed on the pre-planned spraying trajectory sequence to obtain the spraying trajectory sequence, thereby forming the robot trajectory path to be sprayed, which is based on the workpiece coordinate system. Thus, the spraying trajectory sequence is obtained.

[0090] Step S4, using the spray trajectory sequence and IO signal to control the physical spray station to spray the product; when spraying, using the real-time communication channel to obtain data in the robot arm controller to optimize the spray trajectory.

[0091] Real-time control of the robot's spraying process relies on communication between the robot's program and I / O signals. This step applies the standard JSON data format used in network communication to represent the robot's trajectory sequence, constructing JSON-formatted data for the spray trajectory sequence and I / O signals.

[0092] Coordinate transformation. The trajectory points of the spraying trajectory sequence calculated in step S3 are trajectory points in the workpiece coordinate system. To apply them to the robot control system, the trajectory points need to be converted into the robot-related coordinate system, such as the robot base coordinate system, tool coordinate system, etc. The coordinate transformation principle is used. B The coordinates in the coordinate system are expressed as B P, transformed into O A The coordinate system is represented as A P, is the rotation matrix of the B coordinate system relative to the A coordinate system, A P B0 Yes O B The coordinates of the origin of the coordinate system in the A coordinate system are expressed as follows:

[0093] The JSON format for representing spray trajectory sequences is characterized by low bandwidth usage, ease of client reading, and parsing support for multiple server programming languages, including C, C++, and Java. Using the JSON data format to represent robot trajectory sequences reduces data transmission bandwidth and is compatible with various robot control systems. The robot trajectory sequence representation data format is divided into trajectory description and I / O signal description, as shown in the following table.

[0094] Table 3 Description of spray trajectory sequence

[0095]

[0096]

[0097] The instruct robot control instructions include those shown in Table 4.

[0098] Table 4instruct robot control instructions

[0099] Serial number Field type describe 1 MoveC string TCP circular motion 2 MoveJ string Joint movement 3 MoveL string TCP linear motion 4 MoveAbsJ string Absolute angular motion of the axis 5 MoveExtJ string External linear and rotary axis motion

[0100] Table 5IO signal description

[0101] Serial number Field type describe 1 variable string IO signal variable name 2 value int IO setting value

[0102] The IO signal variable names include start spraying (start), end spraying (end), and spraying process parameters, such as nozzle diameter adjustment (changeL), spray width adjustment (changeF), and paint spray volume adjustment (changeC). For example, if a 6-axis robotic arm is set to the joint pose {10, 20, 30, 40, 50, 60} and spraying begins, with a spray width of 5mm, the following JSON format data is generated.

[0103] {

[0104] location:{

[0105] id:p1,

[0106] type:joint

[0107] value:[10,20,30,40,50,60],

[0108] instruct:moveJ

[0109] },

[0110] DO:{

[0111] variable:start,

[0112] value:1

[0113] }

[0114] DO:{

[0115] variable:changeF,

[0116] value: 5

[0117] }

[0118] }

[0119] Establish a control program automatic generation module. The control program automatic generation module automatically builds the robot control program according to the robot control statements of different robot control systems. The ABB robot control statement is constructed based on the above JSON format data as follows:

[0120] MoveJ 10,20,30,40,50,60

[0121] Set DO IO1 = 1

[0122] SetDO IO2=5

[0123] Then the physical spraying station is controlled to spray the product.

[0124] During the spraying process, a real-time monitoring trajectory optimization mechanism is established. Using the real-time communication between the virtual spraying station and the physical spraying station's robotic arm in step S2, the virtual spraying station obtains data from the physical spraying station's robotic arm controller in real time, including joint coordinates and displacement coordinates, and can monitor the robotic arm's spraying trajectory in real time. The trajectory error is calculated using the following steps, and automated calibration is performed. The calibration data is output to the robotic arm controller, thereby achieving real-time optimization of the spraying trajectory. The specific steps are as follows:

[0125] a. Use the forward kinematics of the robotic arm to calculate the end position coordinates N of the real-time joint angles of the robotic arm at the first N trajectory points x , N y , N z , alt, belt, gama;

[0126] b. Using the Euclidean distance calculation method, calculate the absolute error between the N real-time end position coordinates of the robotic arm in the previous step and the N trajectory points of the spray trajectory sequence;

[0127] c. Threshold the absolute error to determine whether to perform error optimization. If optimization is required, further use the average deviation calculation method to calculate the deviation x', y', z', alt', belt', gama';

[0128] d. Perform deviation calibration on the next N trajectory positions, and use inverse kinematics to calculate the inverse kinematic solution of the calibrated N trajectory points, and output the calibrated data values ​​to the robot arm controller.

[0129] Finally, build a virtual simulation system based on the BS architecture, including:

[0130] A bidirectional data channel is established between the virtual simulation display and the simulation planning runtime. The virtual simulation display and simulation planning runtime are encapsulated and separated into modules, and data is transmitted between the two modules via a bidirectional TCP channel. Transmitted data includes 3D model translation and rotation matrices, real-time robotic arm joint data, virtual simulation trajectory coordinate data, and 3D point cloud data. This architectural split allows the computationally demanding simulation planning runtime to be encapsulated and centrally deployed on the server, better utilizing the server's high-performance parallel computing capabilities.

[0131] In summary, the spraying operation planning method of the present invention can adapt to the different postures of different workpieces, adapt to the real-time control of robots, and can use the model library and driver library to quickly build a virtual simulation application for spraying operations, remotely monitor the entire spraying process, and perform real-time trajectory optimization. The present invention has achieved high-efficiency production and reduced labor costs from many aspects. The simulation algorithm of the BS architecture can more efficiently utilize the server computing power, greatly improve the simulation speed, and support multi-user collaboration for virtual simulation trajectory planning.

[0132] The present invention first constructs a virtual physical model mapping, establishes a robotic arm model, a nozzle model, a peripheral equipment model and a product model, and uses a CAD file mapping method to assemble and construct a spray processing unit. Then, real-time communication is established, and real-time communication between the virtual simulation and the robotic arm and the spray station master station is established. Then, data binding is performed to quickly construct a data mapping between the physical equipment and the virtual model. 3D point cloud technology is used to match the workpiece posture, and the type of sprayed product is obtained in real time by docking with the production management system. The pre-planned spray trajectory is mapped to construct the spray trajectory of the workpiece to be sprayed. Finally, the trajectory path is output, and the robot trajectory spraying code in the robot coordinate system is output, thereby realizing the spraying operation trajectory planning, and the spraying process of the spray robot can be monitored in real time. The spraying operation planning method of the present invention can adapt to the different postures of different workpieces, adapt to the real-time control of the robot, quickly build a virtual simulation application for the spraying operation, remotely monitor the entire spraying process, and perform real-time trajectory optimization. At the same time, the virtual simulation of the present invention adopts a BS architecture mode. Compared with the traditional CS local deployment, the simulation algorithm of the BS architecture can more efficiently utilize the server computing power, which can greatly improve the simulation speed and support multi-user collaboration for virtual simulation trajectory planning. The present invention has achieved high-efficiency production and reduced labor costs from many aspects. It should be noted that the above-mentioned sequence of the embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. In addition, the process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-tasking and parallel processing are also possible or may be advantageous.

[0133] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0134] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for planning spraying operations for scattered steel workpieces based on virtual simulation, characterized in that: The method includes: Modeling the spraying robot arm, the nozzle tool, the peripheral equipment and the spraying product respectively to obtain a virtual spraying robot arm model, a virtual nozzle tool model, a virtual peripheral equipment model and a virtual spraying product model; assembling the virtual spraying robot arm model, the virtual nozzle tool model, the virtual peripheral equipment model and the virtual spraying product model to obtain a virtual spraying station; Establish a real-time communication channel between the virtual spraying station and the physical spraying station, and bind the virtual spraying station with the physical spraying station; Set a pre-planned spraying trajectory sequence; obtain point cloud data of a physical product and a virtual spraying product model respectively, recorded as first point cloud data and second point cloud data; obtain a final rotation and translation matrix using the first point cloud data and the second point cloud data; execute the final rotation and translation matrix on the pre-planned spraying trajectory sequence to obtain a spraying trajectory sequence; The spraying trajectory sequence and IO signal are used to control the physical spraying station to spray the product; when spraying, the real-time communication channel is used to obtain data in the robot arm controller to optimize the spraying trajectory.

2. According to the method for planning spraying operations of scattered steel workpieces based on virtual simulation in claim 1, it is characterized in that: The method of obtaining a virtual spraying robot arm model comprises: Obtaining a virtual spraying robot arm model includes motion joint hierarchical modeling, motion joint coordinate system origin modeling and zero-point posture calibration.

3. According to the virtual simulation-based method for planning spraying operations for scattered steel workpieces in claim 1, it is characterized in that: The virtual nozzle tool model includes: The nozzle tool is three-dimensionally visualized and modeled, and the nozzle tool obtained by the three-dimensional visualization modeling is bound to the end of the virtual spraying robot arm model; the nozzle process parameters are bound to the nozzle tool obtained by the three-dimensional visualization modeling to obtain a virtual nozzle tool model.

4. According to the virtual simulation-based method for planning spraying operations for scattered steel workpieces in claim 1, it is characterized in that: The virtual spraying robot arm model, the virtual spray head tool model, the virtual peripheral equipment model and the virtual spraying product model are assembled to obtain a virtual spraying station, including: According to the CAD layout file, the virtual spraying robot model, the virtual nozzle tool model, the virtual peripheral equipment model and the virtual spraying product model are assembled to obtain a virtual spraying station, wherein the assembly constraint methods include fixed constraint, coaxial constraint, angle constraint and contact constraint.

5. According to the virtual simulation-based method for planning spraying operations for scattered steel workpieces in claim 1, it is characterized in that: The real-time communication channel between the virtual spraying station and the physical spraying station is established, including: Establishing a real-time communication channel between the virtual spraying station and the physical spraying station includes establishing real-time communication between the virtual spraying station and the mechanical arm of the physical spraying station, and establishing real-time communication between the virtual spraying station and the physical spraying station master station executable logic controller.

6. The method for planning spraying operations for scattered steel workpieces based on virtual simulation according to claim 1 is characterized in that: The method of obtaining a final rotation and translation matrix by using the first point cloud data and the second point cloud data includes: The first point cloud data and the second point cloud data are roughly aligned to obtain an initial rotation and translation matrix; based on the ICP algorithm, the second point cloud data is initially rotated and translated, and the upper limit of the number of iterations based on the ICP algorithm and the error threshold parameter are set; for each point in the first point cloud data, the nearest point is found in the second point cloud data as the corresponding point; the mean square error is used to calculate the error measure between the second point cloud data after transformation and the first point cloud data; the least squares method is used to solve the optimal rotation matrix and translation matrix based on the corresponding point pairs to minimize the error; the obtained rotation and translation matrix is ​​applied to the point cloud to be aligned, and its position is updated to make it close to the reference point cloud; iterative optimization is repeated, and after each iteration, it is checked whether the error is less than the set error threshold parameter, and if so, the final rotation and translation matrix is ​​calculated.

7. The method for planning spraying operations for scattered steel workpieces based on virtual simulation according to claim 6 is characterized in that: The obtaining of an initial rotation and translation matrix by performing rough registration according to the first point cloud data and the second point cloud data comprises: Based on the RANSAC search method, the threshold delt is set, a matching point set is constructed, 3 pairs of matching points are selected and put into the matching point set, the rotation and translation transformation matrix between the matching points is calculated, and the rotation and translation transformation matrix is ​​applied to the second point cloud data to calculate the Euclidean distance between the two point clouds. If it is less than the threshold delt, the matching point set is updated, otherwise, matching points are continued to be selected to supplement the matching points, and the number of iterations N is set to obtain the initial rotation and translation matrix.

8. The method for planning spraying operations for scattered steel workpieces based on virtual simulation according to claim 1 is characterized in that: The method of controlling the physical spraying station by using the spraying trajectory sequence and the IO signal to spray the product includes: Construct the JSON format data of the spraying trajectory sequence and IO signal; construct the ABB robot control statement based on the JSON format data, and use the ABB robot control statement to control the physical spraying station to spray the product.

9. The method for planning spraying operations for scattered steel workpieces based on virtual simulation according to claim 1, characterized in that: The method of using the real-time communication channel to obtain data in the robot arm controller to optimize the spraying trajectory includes: The end position coordinates N of the real-time joint angles of the robot arm at the first N trajectory points are calculated using the robot arm forward kinematics. x , N y , N z , alt, belt, gama; The Euclidean distance calculation method is used to calculate the absolute error between the real-time end position coordinates of the N robotic arms and the N trajectory points of the spraying trajectory sequence; The absolute error is thresholded to determine whether error optimization is required. If optimization is required, the average deviation calculation method is used to calculate the deviation x', y', z', alt', belt', gama'; The deviation of N trajectory positions is calibrated, and the inverse kinematics is used to calculate the inverse kinematics of the calibrated N trajectory points, and the calibrated data values ​​are output to the robot controller to optimize the spraying trajectory.

Images