Robot welding medium plate path planning method and system based on three-dimensional visual perception

Abstract

The invention discloses a robot welding medium plate path planning method and system based on three-dimensional visual perception, and the method comprises the steps: scanning a medium plate workpiece from different angles and heights, and obtaining the three-dimensional point cloud data of the workpiece; carrying out voxel down-sampling processing on the three-dimensional point cloud data of the medium-thickness plate workpiece to obtain processed data; carrying out point cloud target detection on the processed data to obtain spatial position information of the welding path endpoints of the medium-thickness plate; converting the spatial position information into welding point coordinate information under a robot tool coordinate system; and generating a robot motion control script based on the welding point coordinate information. According to the method, the welding path planning efficiency and the adaptability of the robot to the medium-thickness plate workpiece with the complex welding seam type can be remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of robot intelligent welding, and in particular to a method and system for robot welding medium and thick plates path planning based on three-dimensional visual perception. Background Art

[0002] As an important innovation in the manufacturing industry, robot intelligent welding technology is an important indicator of the level of intelligent manufacturing. At present, most robot welding work mainly relies on cumbersome manual teaching reproduction and offline programming methods, which is still far from the ideal level of robot intelligent welding.

[0003] Robots face many technical difficulties when welding medium and thick plate workpieces with complex structures through manual teaching and offline programming. Such workpieces are usually composed of medium and thick plates with a material thickness greater than 4 mm, and have characteristics such as large spatial dimensions, discrete distribution of welds, and complex weld types. This leads to low efficiency and poor adaptability of robot path planning for medium and thick plate welding using methods such as teaching reproduction and offline programming. In actual production processes, manual welding methods are often used to weld complex medium and thick plate components, which has the disadvantages of large welding workload, harsh working environment, and uneven weld quality.

[0004] Therefore, in order to solve the above problems, a robot welding medium and thick plate path planning method and system based on three-dimensional visual perception is needed, which can significantly improve the robot's welding path planning efficiency and adaptability for medium and thick plate workpieces with complex weld types. Summary of the invention

[0005] In view of this, the purpose of the present invention is to overcome the defects in the prior art and provide a robot welding medium and thick plate path planning method and system based on three-dimensional visual perception, which can significantly improve the robot's welding path planning efficiency and adaptability for medium and thick plate workpieces with complex weld types.

[0006] The robot welding medium and thick plate path planning method based on three-dimensional visual perception of the present invention comprises:

[0007] Scan the medium and thick plate workpiece from different angles and heights to obtain the three-dimensional point cloud data of the workpiece;

[0008] Preprocess the three-dimensional point cloud data of the medium and thick plate workpiece to obtain processed data;

[0009] Perform point cloud target detection on the processed data to obtain the spatial position information of the end points of the welding path of the medium and thick plates;

[0010] Convert the spatial position information into welding point coordinate information in the robot tool coordinate system;

[0011] Generate robot motion control script based on welding point coordinate information.

[0012] Furthermore, voxel down sampling is used to pre-process the 3D point cloud data of the medium and thick plate workpiece, specifically including:

[0013] Divide the point cloud space into grids to obtain voxelized point cloud space, where each small grid is called a voxel;

[0014] A single centroid point is obtained by weighted averaging multiple points in each small grid and used to replace the points in the original grid.

[0015] Furthermore, target detection is performed on the processed data, including:

[0016] The improved SA-SSD point cloud detection model is used to output the bounding box of the initial position of the welding path, and all the points in these bounding boxes are weighted averaged to obtain the position information of the starting point of each welding path in the camera coordinate system.

[0017] Furthermore, the improved SA-SSD point cloud detection model takes SA-SSD as the baseline model, introduces the TripletAttention mechanism behind each layer of 3D convolutional features in the backbone network; uses EPSANet Block to replace the original 2D convolution in the detection head of the model; and uses the 3DMPDIoU border regression loss function to replace the Smooth-l1 border regression loss function of the original SA-SSD model.

[0018] Furthermore, for the input tensor, TripletAttention first performs a rotation operation and then uses a residual transformation to establish dependencies between dimensions, and extracts inter-channel and spatial information with a negligible amount of parameters;

[0019] TripletAttention consists of three branches. The first branch is used to calculate the attention weights of the channel dimension C and the spatial dimension W. The second branch is used to capture the cross-dimensional interaction between the channel dimension C and the spatial dimension H. The third branch is used to establish spatial attention.

[0020] Finally, the three branches are averaged and summarized to get the output tensor.

[0021] Furthermore, the core module for implementing multi-scale feature extraction in EPSANet Block is the PSA module; the PSA module first constructs multi-scale features through the SPC module; then, it extracts features of different scales from the channel using compression and excitation, and performs Softmax operations on these features; finally, the corrected attention vector is applied to the multi-scale feature map, and the result is used as output.

[0022] Furthermore, the loss function L of the improved SA-SSD point cloud detection model is determined according to the following formula:

[0023] L=L cls ⊕ηL bbox ⊕μL seg ⊕λL ctr ;

[0024] Among them, η=2, μ, λ are the hyperparameters set during the training of the auxiliary network, and ⊕ is the addition operation;

[0025] L cls is the classification loss function:

[0026] L cls =(1-β t ) r ×log(β t );

[0027] Among them, β t It indicates the probability that the prediction result is a positive sample, and r is the coefficient for adjusting positive and negative samples;

[0028] L bbox is the bounding box regression loss function:

[0029] L bbox =1-3DMPDIoU;

[0030]

[0031] ξ 2 =h 2 +c 2 +w 2 ;

[0032] Among them, let the range of the overall point cloud space be (h, c, w), and the vertex coordinates of the real box The model predicts the coordinates of the vertices of the box to be (x i ′ ,y i ′ ,z i ′ );b gt is the set of real boxes, b ′is the set of prediction boxes;

[0033] L seg Loss function for foreground segmentation:

[0034]

[0035] Among them, σ and γ are the hyperparameters of model training, N pos Represents the number of foreground points, Indicates the probability that the point i is in the foreground segmentation area; s i Indicates whether point i falls within the real box;

[0036] L ctr The loss function for center estimation is:

[0037]

[0038] in, is the branch output of center estimation, and Δρ is the offset from the target point to the corresponding center point.

[0039] Further, the spatial position information is converted into welding point coordinate information in the robot tool coordinate system, specifically including:

[0040] Take photos of the a×b chessboard at different positions and angles, and collect u chessboard images; solve the intrinsic parameter matrix in MATLAB to obtain the camera intrinsic parameter matrix;

[0041] By recording the posture of the welding robot when collecting chessboard images at different positions, the rotation vector and translation vector of the chessboard relative to the robot base are obtained;

[0042] Combined with the camera intrinsic parameter matrix, the hand-eye matrix between the camera coordinate system and the welding robot tool coordinate system is solved;

[0043] The spatial position corresponding to the camera coordinates of each end point of the welding path of the medium and thick plate is right-multiplied by the hand-eye matrix to obtain the coordinates of the starting point of the welding path in the robot tool coordinate system.

[0044] A robot welding medium and thick plate path planning system based on three-dimensional visual perception for realizing a robot welding medium and thick plate path planning method, comprising a robot motion execution unit and a three-dimensional visual perception unit;

[0045] The robot motion execution unit includes a welding robot body, a welding gun, a camera bracket and a robot control cabinet; the three-dimensional visual perception unit includes an industrial camera; wherein the end effector of the welding robot body and the industrial camera are installed in an eye-on-hand manner.

[0046] The beneficial effects of the present invention are as follows: the present invention discloses a method and system for robot welding medium and thick plate path planning based on three-dimensional visual perception, uses a 3D industrial camera to perform a global scan of the medium and thick plate workpiece to obtain the workpiece point cloud data, and combines the voxel downsampling method to filter the noise points and discrete points in the point cloud data; designs an improved SA-SSD point cloud detection algorithm to realize the identification of the end points of the medium and thick plate welding path and the output of position information, improves the detection accuracy of the algorithm, converts the camera coordinates of the path end points into three-dimensional coordinates in the robot tool coordinate system, and generates a motion control program and sends it to the welding robot to realize the medium and thick plate welding path planning. The present invention realizes the robot to perform adaptive welding path planning for medium and thick plate workpieces, achieving the effect of "no teaching" and "no programming". BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0048] Figure 1 A schematic diagram of the process flow of medium and thick plate welding path planning of the present invention;

[0049] Figure 2 A scanning schematic diagram of a medium-thick plate structural member of the present invention;

[0050] Figure 3 Schematic diagram of the voxel downsampling process of the present invention;

[0051] Figure 4 It is a schematic diagram of the structure of the improved SA-SSD point cloud detection model of the present invention;

[0052] Figure 5 (a) is a schematic diagram of the initial position detection results of the medium and thick plate welding at the power tower corner;

[0053] Figure 5 (b) is a schematic diagram of the initial position detection results of the medium and thick plate welding of the thrust plate of the plate and frame filter press;

[0054] Figure 6 (a) is a schematic diagram of the end point detection results of the medium and thick plate welding path for the power tower corner;

[0055] Figure 6 (b) is a schematic diagram of the end point detection results of the welding path of the medium and thick plate of the thrust plate of the plate and frame filter press;

[0056] Figure 7 It is a schematic diagram of the structure of the TripletAttention module of the present invention;

[0057] Figure 8 Schematic diagram of the improved model prediction frame and the real frame of the present invention;

[0058] Fig. 9A schematic diagram of chessboard corner point detection according to the present invention;

[0059] Fig.10 It is a schematic diagram of the welding robot motion control program of the present invention. DETAILED DESCRIPTION

[0060] The present invention is further described below in conjunction with the accompanying drawings, as shown in the drawings:

[0061] This embodiment discloses a robot welding medium and thick plate path planning method based on three-dimensional visual perception, comprising the following steps:

[0062] Scan the medium and thick plate workpiece from different angles and heights to obtain the three-dimensional point cloud data of the workpiece;

[0063] Preprocess the three-dimensional point cloud data of the medium and thick plate workpiece to obtain processed data;

[0064] Perform point cloud target detection on the processed data to obtain the spatial position information of the end points of the welding path of the medium and thick plates;

[0065] Convert the spatial position information into welding point coordinate information in the robot tool coordinate system;

[0066] Generate robot motion control script based on welding point coordinate information.

[0067] The present invention adopts a voxel downsampling method to pre-process the point cloud data of medium and thick plate workpieces, solving the influence of data properties on the recognition of medium and thick plate welding path endpoints. Taking full account of the spatial structure and welding process of medium and thick plate workpieces, the robot completes the adaptive welding path planning of medium and thick plate workpieces with complex weld types by scanning the workpiece with a depth camera at multiple angles, improving the 3D point cloud target detection algorithm, and robot hand-eye calibration, solving the problems of difficulty in robot recognition of welding path endpoints of medium and thick plate workpieces and low recognition accuracy. Under the premise of ensuring the accuracy requirements of medium and thick plate welding path planning, the present invention significantly improves the efficiency and adaptability of the robot's welding path planning for medium and thick plate workpieces with complex weld types.

[0068] The present invention also relates to a robot welding medium and thick plate path planning system based on three-dimensional visual perception for realizing a robot welding medium and thick plate path planning method, comprising a robot motion execution unit and a three-dimensional visual perception unit;

[0069] like Figure 1As shown, the robot motion execution unit includes a welding robot body, a welding gun, a camera bracket and a robot control cabinet; the three-dimensional visual perception unit includes an industrial camera; wherein the end effector of the welding robot body and the industrial camera are installed in an eye-on-hand manner. The industrial camera is an industrial surface scanning binocular structured light camera, and an existing 3D industrial camera can be used.

[0070] The HSR-JH605-1400 welding robot is used as the carrier. The robot adopts a hollow structure arm with light weight and compact structure. It can effectively eliminate the interference between the welding gun and cables, medium and thick plate workpieces and peripheral equipment when working in a small space.

[0071] The visual sensor uses Zhixiang Optoelectronics' Surface HD-100 industrial camera. The camera mainly uses a diffraction grating to project infrared laser to illuminate the surface of the object, and receives the reflected light through the camera lens to obtain color images, depth images and point cloud data of medium and thick plate workpieces.

[0072] In this embodiment, the point cloud is a collection of all points of an object in a coordinate system. Compared with image data, point cloud data can directly represent the three-dimensional coordinates (X, Y, Z) of a single point in the coordinate system where the object is located. Figure 2 As shown, a 3D industrial camera is used to perform a global scan of a medium and thick plate workpiece from different angles to obtain point cloud data of the workpiece.

[0073] However, the point cloud data of thick plate workpieces obtained by 3D camera scanning contains a lot of redundant data, which will increase the computational complexity and training time of the point cloud target detection model. In addition, the original point cloud data contains some noise points and discrete points, which will affect the detection of the endpoints of the thick plate welding path. Therefore, the original point cloud data is preprocessed by voxel downsampling.

[0074] Voxel downsampling is used to preprocess the 3D point cloud data of medium and thick plate workpieces, including:

[0075] like Figure 3 As shown, first in the point cloud space Figure 3 (a) is divided into grids to obtain a voxelized point cloud space Figure 3 (b), each small grid is called a voxel. Then, multiple points in each small grid are weighted averaged to obtain a single point, which is used to replace the points in the original grid. The result is as follows Figure 3 (c) as shown.

[0076] The calculation process is as follows: Assume that each voxel contains k points, and the coordinates of each point are (x i ,y i ,z i), i = 1, 2, ..., k, and the weighted average of the voxels is O center (X center ,Y center ,Z center ), then the center of mass of the voxel is:

[0077]

[0078] In this embodiment, the core of the SA-SSD algorithm is to enhance the ability of the backbone network to extract target features by designing an additional auxiliary network. The auxiliary network needs to convert the tensor features extracted by the backbone network into point-level features, and perform joint optimization with the supervised learning method to guide the convolutional features in the backbone network to perceive the point cloud structure of the target. Among them, the most important is the point feature representation. If the point feature set is {(F i ,E i ),i=1,2,…,n},F i Indicates the coordinates as E i The point features of the auxiliary network are optimized as follows: {(F ′ j ,E j ),j=1,2,…,m}, then we have the following relationship:

[0079]

[0080] in:

[0081] From the interpolation method above, we can see that the joint optimization method of the auxiliary network uses distance as the weight for weighting. It means setting the distance range to 0.05, and using the scale of (0.05, 0.05, 0.1) to divide the voxel grid. Finally, the multi-layer voxel grid is concat with the point feature to realize the point feature representation.

[0082] However, the detection accuracy limit of the single-stage 3D target detection algorithm is relatively low. In order to further improve the detection accuracy of the endpoints of the welding path of medium and thick plates, the present invention makes some optimization improvements based on the SA-SSD model. The improved 3D target detection model is the improved SA-SSD point cloud detection model, and its structure is as follows: Figure 4 shown.

[0083] Perform target detection on the processed data, including:

[0084] The improved SA-SSD point cloud detection model is used to output the bounding box of the initial position of the welding path, and all the points in these bounding boxes are weighted averaged to obtain the position information of the starting point of each welding path in the camera coordinate system.

[0085] Among them, the medium and thick plate point cloud data after voxel downsampling is used to make a dataset in kitti format, and the dataset is input into the improved SA-SSD model for training. The deep learning environment used for model training includes PyTorch, MMDetection3D, and OpenCV. The hardware configuration includes Intel(R)Core(TM)i5-35612400F@2.50GHz CPU, a12GB NVIDIA GeForce RTX 3060GPU and a 64GB RAM. Finally, the model file generated by algorithm training is used to detect the initial position of the medium and thick plate welding path. The detection results are as follows Figure 5 shown.

[0086] The improved SA-SSD point cloud detection model can identify the starting position of the welding path in all medium and thick plate workpieces, and the output prediction box can fully cover all target point cloud features. Subsequently, the point cloud in each prediction box is weighted averaged and the three-dimensional coordinates of the end point of the medium and thick plate welding path are output.

[0087]

[0088] f p is the weight coefficient, N represents the index value of the number of points in each prediction box, and the output results of the end points of the medium and thick plate welding path are as follows: Figure 6 shown.

[0089] In this embodiment, the improved SA-SSD point cloud detection model uses SA-SSD as the baseline model, and introduces the TripletAttention mechanism behind each layer of 3D convolution features in the backbone network; EPSANet Block is used to replace the original 2D convolution in the detection head of the model; and the 3DMPDIoU border regression loss function is used to replace the Smooth-l1 border regression loss function of the original SA-SSD model.

[0090] TripletAttention is a new module that calculates attention weights by capturing cross-dimensional interactions using a three-branch structure. For the input tensor, TripletAttention first performs a rotation operation and then uses a residual transformation to establish dependencies between dimensions and extract inter-channel and spatial information with negligible number of parameters. The structure of TripletAttention is shown in Figure 7 shown.

[0091] Depend on Figure 7It can be seen that TripletAttention contains three branches. The first branch is used to calculate the attention weights of the channel dimension C and the spatial dimension W. The second branch is responsible for capturing the cross-dimensional interaction between the channel dimension C and the spatial dimension H. The third branch is used to establish spatial attention. Finally, the three branches are averaged and summarized to obtain the output tensor. Among them, Z_Pool represents the maximum pooling and average pooling operations on the tensor.

[0092] Z Pool (tensor) = [MaxPool 0d (tensor),AvgPool 0d (tensor)]

[0093] In the above formula, 0d represents the 0th dimension for maximum pooling and average pooling.

[0094] In this embodiment, a pyramid separable attention (PSA) module is used to replace the 3×3 convolution in the bottleneck layer of the ResNet network, and an efficient pyramid squeezed attention block (EPSANet) based on a convolutional neural network can be obtained, thereby improving the feature expression ability of the model.

[0095] The core module for multi-scale feature extraction in EPSANet Block is the PSA module. The PSA module first constructs multi-scale features through the SPC module. Then, it extracts features of different scales from the channel by compression and excitation, and performs Softmax operation on these features. Finally, the corrected attention vector is applied to the multi-scale feature map, and the result is used as output.

[0096] The specific calculation process is as follows: Let the input of the SPC module be Q, first split it into S parts, that is, [Q0,Q1,…,Q S-1 ], then extract multi-scale features from different parts, and finally concatenate the extracted multi-scale features through the Concat operation. The specific description is as follows:

[0097]

[0098] In the above formula, K i represents the convolution kernel size of the i-th part, G i represents the Group size of the i-th part, and Concat represents the concatenation of features in the channel dimension. Subsequently, attention weights are extracted for features of different parts, and the formula is as follows:

[0099]

[0100] Finally we get:

[0101]

[0102] Among them, ⊙ represents the multiplication operation on the channel dimension.

[0103] In this embodiment, the loss function L of the improved SA-SSD point cloud detection model is mainly composed of the classification loss function L cls , border regression loss function L bbox , foreground segmentation loss function L seg And the center estimation loss function L ctr composition.

[0104] L cls is the classification loss function:

[0105] L cls =(1-β t ) r ×log(β t );

[0106] Among them, β t It indicates the probability that the prediction result is a positive sample, and r is the coefficient for adjusting positive and negative samples;

[0107] The improved model uses the 3DMPDIoU loss function to optimize the bounding box regression loss. Figure 8 As shown in Figure 2, the core idea of ​​3DMPDIoU is to approximate each bounding box predicted by the model to its corresponding real box. The specific calculation method is as follows:

[0108] according to Figure 8 It can be seen that, assuming the range of the overall point cloud space is (h, c, w), the vertex coordinates of the real box The model predicts the coordinates of the vertices of the box to be (x i ′ ,y i ′ ,z i ′ ), then the calculation method of 3DMPDIoU is as follows:

[0109]

[0110] In the above formula, b gt is the set of real boxes, b ′ is the set of predicted boxes. Therefore, the calculation method of the bounding box regression loss function of the improved SA-SSD model is as follows:

[0111] L bbox is the bounding box regression loss function:

[0112]

[0113] in, Representative n In the norm form, the mean square error (MSE) is used here, and Ω is the parameter of the deep regression model.

[0114] L seg Loss function for foreground segmentation:

[0115]

[0116] Among them, σ and γ are the hyperparameters of model training, which are generally set to 0.25 and 2. pos Represents the number of foreground points, Indicates the probability that the point i is in the foreground segmentation area; s i Indicates whether point i falls within the real box, 1 means it is within, 0 means it is not within;

[0117] The center estimation loss function adopts a variant of the Smoothl1 loss function, and its calculation process is as follows:

[0118] L ctr The loss function for center estimation is:

[0119]

[0120] in, is the branch output of center estimation, and Δρ is the offset from the target point to the corresponding center point.

[0121] In summary, the loss function L of the improved SA-SSD point cloud detection model is:

[0122] L=L cls ⊕ηL bbox ⊕μL seg ⊕λL ctr ;

[0123] Among them, η=2, μ, λ are the hyperparameters set during the training of the auxiliary network, and ⊕ is the addition operation.

[0124] In this embodiment, in order to allow the welding robot to obtain the position and posture of the target position in the robot coordinates, the eye-on-hand hand-eye calibration method is used to solve the conversion relationship between the camera coordinate system and the robot tool coordinate system. Since the present invention adopts the eye-on-hand camera installation method, the change relationship between the camera coordinate system and the robot tool coordinate system is fixed, which is also the quantity to be solved. The spatial position information is converted into the welding point coordinate information in the robot tool coordinate system, that is, the coordinates of the starting point of each welding path in the robot tool coordinate system are calculated using the solved hand-eye conversion matrix.

[0125] The main steps to solve the hand-eye conversion matrix include the following two parts: camera internal and external parameter calibration, and hand-eye calibration.

[0126] First, the welding robot was controlled to take pictures of a 10×7 chessboard at different positions and angles, and 20 chessboard images were collected. The camera intrinsic calibration can use the intrinsic calibration technology, which will not be described here. Then, the intrinsic matrix is ​​solved in the camera calibration module in MATLAB, and the obtained intrinsic matrix is ​​as follows:

[0127]

[0128] Among them, f x and f y represents the focal length of the camera in the x and y directions, c x and c y Indicates the position of the camera's optical center in the x and y directions.

[0129] By recording the posture of the welding robot when collecting chessboard images at different positions, the rotation vector and translation vector of the chessboard relative to the robot base are obtained. Then, the hand-eye matrix between the camera coordinate system and the welding robot tool coordinate system is solved by combining the camera intrinsic parameter matrix obtained previously. Among them, the present invention uses the Hand-eye Calibration function in Opencv to implement the calculation process of the hand-eye matrix. During the hand-eye calibration process, the corner point detection of the chessboard is as follows Fig. 9 shown.

[0130] The calculated hand-eye matrix from the camera coordinate system to the welding robot tool coordinate system is as follows:

[0131]

[0132] Finally, the camera coordinates of each end point of the welding path of the medium and thick plate are right-multiplied by the hand-eye matrix to obtain the coordinates P of the starting point of the welding path in the robot tool coordinate system:

[0133]

[0134] In this embodiment, after obtaining the welding path starting point coordinates P in the robot tool coordinate system, these point information are transmitted to the industrial computer through the IPC protocol and saved in the position register under the industrial computer for easy reading when generating the robot welding path planning program.

[0135] Finally, the welding robot motion control program is generated by calling the HSC3Api interface and the welding path starting point data, and then sent to the welding robot controller to realize the medium and thick plate welding path planning. Fig.10 shown.

[0136] The present invention provides an objective and scientific path planning method and system for robot welding of medium and thick plates. The original SA-SSD model has the slowest convergence speed and the worst convergence effect, followed by the method with the addition of TripletAttention and EPSANet modules, while the loss function of the present invention has the fastest convergence speed and the best effect. In terms of the average accuracy performance of the model, the average accuracy of the improved method of the present invention is higher than that of other methods, which shows that the improved method is effective in improving the performance of the three-dimensional target detection model.

[0137] The distance error between the coordinates of the welding path endpoints obtained by the present invention and the welding gun front end coordinates corresponding to the welding path endpoints obtained by the manual teaching method is less than 1.16 mm, and the average distance error is controlled at about 1.04 mm, which meets the error requirements for medium and thick plate welding. Therefore, the method of the present invention can replace the manual teaching method to complete the task of medium and thick plate welding path planning. Compared with the traditional teaching and reproduction method, the efficiency of the medium and thick plate welding path planning of the present invention has been significantly improved, and the medium and thick plate welding path planning has small error, high efficiency, strong adaptability, and has excellent industrial application prospects.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A robot welding medium and thick plate path planning method based on three-dimensional visual perception, characterized in that: include: Scan the medium and thick plate workpiece from different angles and heights to obtain the three-dimensional point cloud data of the workpiece; Preprocess the three-dimensional point cloud data of the medium and thick plate workpiece to obtain processed data; Perform point cloud target detection on the processed data to obtain the spatial position information of the end points of the welding path of the medium and thick plates; Convert the spatial position information into welding point coordinate information in the robot tool coordinate system; Generate robot motion control script based on welding point coordinate information.

2. The path planning method for robot welding of medium and thick plates based on three-dimensional visual perception according to claim 1 is characterized in that: Voxel downsampling is used to preprocess the 3D point cloud data of medium and thick plate workpieces, including: Divide the point cloud space into grids to obtain voxelized point cloud space, where each small grid is called a voxel; A single centroid point is obtained by weighted averaging multiple points in each small grid and used to replace the points in the original grid.

3. The path planning method for robot welding of medium and thick plates based on three-dimensional visual perception according to claim 1 is characterized in that: Perform target detection on the processed data, including: The improved SA-SSD point cloud detection model is used to output the bounding box of the initial position of the welding path, and all the points in these bounding boxes are weighted averaged to obtain the position information of the starting point of each welding path in the camera coordinate system.

4. The path planning method for robot welding of medium and thick plates based on three-dimensional visual perception according to claim 3 is characterized in that: The improved SA-SSD point cloud detection model uses SA-SSD as the baseline model, introduces the TripletAttention mechanism behind each layer of 3D convolutional features in the backbone network; uses EPSANet Block to replace the original 2D convolution in the detection head of the model; and uses the 3DMPDIoU border regression loss function to replace the Smooth-l1 border regression loss function of the original SA-SSD model.

5. The path planning method for robot welding of medium and thick plates based on three-dimensional visual perception according to claim 4 is characterized in that: For the input tensor, TripletAttention first performs a rotation operation and then uses a residual transformation to establish dependencies between dimensions and extract inter-channel and spatial information with negligible number of parameters. TripletAttention consists of three branches. The first branch is used to calculate the attention weights of the channel dimension C and the spatial dimension W. The second branch is used to capture the cross-dimensional interaction between the channel dimension C and the spatial dimension H. The third branch is used to establish spatial attention. Finally, the three branches are averaged and summarized to get the output tensor.

6. The method for robot welding medium and thick plates based on three-dimensional visual perception according to claim 4 is characterized in that: The core module for multi-scale feature extraction in EPSANet Block is the PSA module. The PSA module first constructs multi-scale features through the SPC module. Then, it extracts features of different scales from the channel by compression and excitation, and performs Softmax operation on these features. Finally, the corrected attention vector is applied to the multi-scale feature map, and the result is used as output.

7. The path planning method for robot welding of medium and thick plates based on three-dimensional visual perception according to claim 4 is characterized in that: The loss function L of the improved SA-SSD point cloud detection model is determined according to the following formula: Among them, η=2, μ, λ are the hyperparameters set during the training of the auxiliary network. is the addition operation; L cls is the classification loss function: L cls =(1-β t ) r ×log(β t ); Among them, β t It indicates the probability that the prediction result is a positive sample, and r is the coefficient for adjusting positive and negative samples; L bbox is the bounding box regression loss function: L bbox =1-3DMPDIoU; ξ 2 =h 2 +c 2 +w 2 ; Among them, let the range of the overall point cloud space be (h, c, w), and the vertex coordinates of the real box The model predicts the coordinates of the vertices of the box to be (x i ′ ,y i ′ ,z i ′ );b gt is the set of real boxes, b ′ is the set of prediction boxes; L seg Loss function for foreground segmentation: Among them, σ and γ are the hyperparameters of model training, N pos Represents the number of foreground points, Indicates the probability that the point i is in the foreground segmentation area; s i Indicates whether point i falls within the real box; L ctr The loss function for center estimation is: in, is the branch output of center estimation, and Δρ is the offset from the target point to the corresponding center point.

8. The path planning method for robot welding of medium and thick plates based on three-dimensional visual perception according to claim 1 is characterized in that: The spatial position information is converted into the welding point coordinate information in the robot tool coordinate system, including: Take photos of the a×b chessboard at different positions and angles, and collect u chessboard images; solve the intrinsic parameter matrix in MATLAB to obtain the camera intrinsic parameter matrix; By recording the posture of the welding robot when collecting chessboard images at different positions, the rotation vector and translation vector of the chessboard relative to the robot base are obtained; Combined with the camera intrinsic parameter matrix, the hand-eye matrix between the camera coordinate system and the welding robot tool coordinate system is solved; The spatial position corresponding to the camera coordinates of each end point of the welding path of the medium and thick plate is right-multiplied by the hand-eye matrix to obtain the coordinates of the starting point of the welding path in the robot tool coordinate system.

9. A robot welding medium and thick plate path planning system based on three-dimensional visual perception that implements the robot welding medium and thick plate path planning method according to any one of claims 1 to 8, characterized in that: It includes a robot motion execution unit and a three-dimensional visual perception unit; The robot motion execution unit includes a welding robot body, a welding gun, a camera bracket and a robot control cabinet; the three-dimensional visual perception unit includes an industrial camera; wherein the end effector of the welding robot body and the industrial camera are installed in an eye-on-hand manner.

Images