Steel structure three-dimensional workpiece welding seam identification and path planning method and system

Abstract

The invention discloses a steel structure three-dimensional workpiece welding seam identification and path planning method and system, and relates to the technical field of robot intelligent welding, and the method comprises the steps: building a multi-view fusion point cloud corresponding to a target workpiece; performing welding seam segmentation on the multi-view fusion point cloud by using a welding seam neighborhood point cloud segmentation model to obtain a multi-view fusion welding seam neighborhood point cloud; performing point cloud plane fitting on the multi-view fusion welding seam neighborhood point cloud to obtain a plurality of target welding seam neighborhood planes; and determining a corresponding welding seam straight line based on each pair of intersected target welding seam neighborhood planes, and identifying a welding path and a welding posture according to the welding seam straight line. The method provided by the invention shows good applicability under a complex background, has the advantages of automation, high efficiency, high recognition precision, strong adaptability and the like, and meets the intelligent and flexible operation requirements of the welding robot.

Description

Technical Field

[0001] The present invention relates to the field of intelligent welding technology, and in particular to a method and system for weld recognition and path planning of a three-dimensional steel structure workpiece. Background Art

[0002] As a basic structural connection technology, welding plays a vital role in many fields such as automobile manufacturing, construction transportation, and aerospace. With the rapid development of automation technology, welding robots have been used more and more widely, greatly improving the efficiency and consistency of the welding process. However, in practical applications, the operation modes of welding robots are still mainly "teaching playback" and "offline programming". The "teaching playback" mode relies on manual teaching, and the operator needs to manually guide the robot to complete the setting of the welding path at the welding site. This method will produce errors in complex workpieces or changing environments and lacks flexibility. At the same time, the "offline programming" mode requires the welding path to be generated in advance. Although it can improve efficiency, the model creation and adjustment process is often cumbersome and time-consuming for complex welding tasks. In addition, the two modes of "teaching playback" and "offline programming" are poor in fault tolerance and anti-interference, and cannot effectively cope with changes in the welding process, such as slight deviations of the material or interference from the external environment.

[0003] In the existing technology, intelligent welding technology based on visual sensing is mainly used in relatively simple scenarios, and it is difficult to cope with the diversity of welding scenarios, the uncertainty of the number of welds, and the complexity of welding workpieces. Therefore, it is particularly important to improve the adaptability of welding robots in complex environments and develop more intelligent welding systems. It is necessary to propose a steel structure three-dimensional workpiece weld recognition and path planning method and system to solve or at least alleviate some of the above defects. Summary of the invention

[0004] The main purpose of the present invention is to provide a method and system for weld recognition and path planning of three-dimensional steel structure workpieces, aiming to solve the technical problem in the prior art that it is difficult to accurately identify the weld position based on machine vision, resulting in poor adaptability of welding robots in complex environments.

[0005] To achieve the above-mentioned object, the present invention provides a method for weld identification and path planning of a three-dimensional steel structure workpiece, comprising the steps of: S10, using point cloud splicing and registration technology to establish a multi-view fusion point cloud corresponding to a target workpiece based on multiple structural point cloud data sets; S20, using a weld neighborhood point cloud segmentation model to perform weld segmentation on the multi-view fused point cloud to obtain a multi-view fused weld neighborhood point cloud; S30, performing point cloud plane fitting on the multi-view fused weld neighborhood point cloud to obtain multiple target weld neighborhood planes, specifically including: S31, processing the current multi-view fused weld neighborhood point cloud to obtain a single-plane weld neighborhood point cloud with the largest number of internal points as the target surface weld neighborhood point cloud; S32, determining whether the current number of inner points of the target surface weld neighborhood point cloud is lower than a preset point number threshold; S33, if the current number of inner points of the target surface weld neighborhood point cloud is not less than the preset point number threshold, the target surface weld neighborhood point cloud is removed from the multi-view fused weld neighborhood point cloud to obtain the remaining weld point cloud, the remaining weld point cloud is updated to the multi-view fused weld neighborhood point cloud and the process proceeds to step S31; S34, repeating steps S31 to S33 until the current number of internal points of the target surface weld neighborhood point cloud is lower than a preset point number threshold, and obtaining all target surface weld neighborhood point clouds in the multi-view fusion weld neighborhood point cloud; After determining the target surface weld neighborhood point cloud, the target surface weld neighborhood point cloud is plane optimized to obtain the target weld neighborhood plane; S40, determining a corresponding welding seam straight line based on each pair of intersecting target weld neighborhood planes, and identifying a welding path and a welding posture according to the welding seam straight line.

[0006] Furthermore, the steps of obtaining a single single-plane weld neighborhood point cloud include: Randomly select three sampling points from the multi-view fusion weld neighborhood point cloud, and calculate the plane normal vectors of the sampling plane formed by the three sampling points, where the distance between the sampling points is less than a preset sampling threshold; Calculate the plane parameters of the sampling plane and determine the plane equation; Calculate the distance from all points to the sampling plane and use the error function to evaluate and determine the inliers, where points with distances less than the preset distance threshold are considered inliers: Repeat the above steps until the number of inliers and / or the number of iterations meet the confidence condition.

[0007] Furthermore, the steps of performing plane optimization include: Calculate the The centroid of the target surface weld neighborhood point cloud is obtained by establishing a KD tree to query the centroid neighborhood point cloud; Two sampling points and the centroid are randomly selected on the extracted centroid neighborhood point cloud for plane fitting to obtain the target weld neighborhood plane.

[0008] Furthermore, the steps of establishing the weld neighborhood point cloud segmentation model include: S201, obtaining a workpiece point cloud training data set corresponding to the training workpiece; S202, using a data enhancement method to expand the number of workpiece point cloud training data sets, and using a principal component analysis method to calculate normal vector information of each training point cloud data in the expanded workpiece point cloud training data set; S203, annotating the workpiece weld region of interest in the expanded workpiece point cloud training data set with a weld label, and establishing a weld neighborhood point cloud segmentation model by training the workpiece point cloud training data set annotated with weld labels containing normal vector information, wherein the width of the workpiece weld region of interest is within a width threshold range.

[0009] Furthermore, the weld neighborhood point cloud segmentation model includes a feature extraction module, a cross attention module, and multiple fully connected layers. The feature extraction module is a U-shaped structure based on PointNet and PointNet++; The feature extraction module uses set abstraction and feature propagation to predict the point cloud semantic features of the training point cloud data. ; The point cloud semantic features are transformed into Input into the shared multi-layer perceptron to obtain the key feature map , query feature graph and value feature graph ; Using formula Calculate the output feature map , where i∈M, Softmax is a normalized exponential function; , and is matrix addition, subtraction, and multiplication; A mapping relationship between the output feature map and the training point cloud data annotated with weld labels is established through the fully connected layer to establish a weld neighborhood point cloud segmentation model to achieve classification or regression of point cloud features; During the model training process, the cross entropy loss function is used to evaluate the difference between the model prediction value and the actual labeled value to optimize the model.

[0010] Further, step S40 also includes identifying a welding path and a welding posture according to the welding weld straight line; Step S40 specifically includes: S41, determining a weld intersection line and a weld centerline vector formed by the intersection of each pair of intersecting target weld neighborhood planes; S42, extracting point clouds in the neighborhood of the weld intersection line by using a clustering algorithm, and projecting all point clouds inside the cluster onto the weld intersection line; S43, sorting the projection point set obtained by projection, and determining the starting projection point and the ending projection point; S44, the weld segment between the starting projection point and the ending projection point is sampled with a fixed step size to obtain a discrete point set ; S45, establish a discrete coordinate system relative to the robot base coordinate system at each discrete point, wherein the x-axis of the discrete coordinate system is determined by the weld direction vector, the z-axis is determined by the weld centerline vector, and the y-axis is determined by the right-hand rule; use the direction vectors of the x-axis, y-axis and z-axis to construct a rotation matrix to determine the welding path and welding posture of the multi-degree-of-freedom robot.

[0011] Furthermore, using the formula Determine the weld centerline vector ,in, , , , are the normal vectors of a pair of intersecting target weld neighborhood planes, and d is the direction vector of the weld intersection line.

[0012] Furthermore, step S10 specifically includes: S11, using a coded structured light sensor to obtain a target structure point cloud dataset of the target workpiece under multiple viewing angles, wherein one viewing angle corresponds to one structure point cloud dataset; S12, transforming the local point cloud in each structure point cloud data set into the robot base coordinate system through a rotation matrix, wherein the rotation matrix is ​​a transformation matrix mapping the sensor coordinate system to the robot base coordinate system; S13, combining the k-means clustering algorithm and the iterative closest point algorithm to align the local point clouds after coordinate transformation, and obtain the multi-view fusion points formed by the fusion of all local point clouds.

[0013] The present invention also provides a steel structure three-dimensional workpiece weld seam recognition and path planning system, a welding robot and a coded structured light sensor, the welding robot has a base and a manipulator, the coded structured light sensor is arranged on the manipulator, and a processing device is arranged on the welding robot, and the processing device is used to implement the steps of the above-mentioned steel structure three-dimensional workpiece weld seam recognition and path planning method.

[0014] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned steel structure three-dimensional workpiece weld identification and path planning method are implemented.

[0015] Compared with the prior art, the steel structure three-dimensional workpiece weld identification and path planning method provided by the present invention has the following beneficial effects: The present invention provides a weld seam recognition and path planning method for a three-dimensional steel structure workpiece. First, a multi-perspective fusion point cloud corresponding to a target workpiece is established. Then, a weld seam neighborhood point cloud segmentation model is used to perform weld seam segmentation on the multi-perspective fusion points to obtain a multi-perspective fusion weld seam neighborhood point cloud. The weld seam neighborhood segmentation model can extract the weld seam area for independent analysis to avoid the influence of complex background and other environmental factors. Then, the multi-perspective fusion weld seam neighborhood point cloud is plane-fitted and the corresponding weld seam straight line is determined based on each pair of intersecting target weld seam neighborhood planes. The plane fitting method proposed in the present invention has faster fitting speed and higher accuracy, and can avoid interference between adjacent planes. The weld seam recognition and path planning method for a three-dimensional steel structure workpiece provided by the present invention can determine the welding path and welding posture without the CAD model of the target weldment (target workpiece), can be applied to the full-process intelligent welding of any complex three-dimensional workpiece composed of multiple planes, and shows good applicability under complex backgrounds. It has the advantages of automatic efficiency, high recognition accuracy and strong adaptability, and meets the intelligent and flexible operation requirements of welding robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0017] Figure 1 A schematic diagram of a flow chart of a method for weld identification and path planning of a three-dimensional steel structure workpiece in one embodiment of the present invention; Figure 2 1 is a schematic diagram of a process for obtaining a plurality of target surface weld neighborhood point clouds in one embodiment of the present invention; Figure 3 is a schematic diagram of a structural point cloud of a target workpiece in a single viewing angle in one embodiment of the present invention; Figure 4 is a schematic diagram of a multi-view fusion point cloud of a target workpiece in one embodiment of the present invention; Figure 5 Schematic diagram of multi-view fusion of weld neighborhood point clouds in one embodiment of the present invention.

[0018] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0019] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0022] In addition, the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0023] Please refer to the attached Figure 1 , Figure 2 , Figure 3 , Figure 4 as well as Figure 5The present invention provides a method for weld identification and path planning of a three-dimensional steel structure workpiece, comprising the following steps: S10, using point cloud splicing and registration technology based on multiple structural point cloud data sets to establish a multi-perspective fusion point cloud corresponding to a target workpiece; S20, using a weld neighborhood point cloud segmentation model to perform weld segmentation on multi-perspective fusion points to obtain a multi-perspective fusion weld neighborhood point cloud; S30, performing point cloud plane fitting on the multi-perspective fusion weld neighborhood point cloud to obtain multiple target weld neighborhood planes, specifically comprising: S31, processing the current multi-perspective fusion weld neighborhood point cloud to obtain a single-plane weld neighborhood point cloud with the largest number of inner points as the target surface weld neighborhood point cloud; S32, judging whether the current number of inner points of the target surface weld neighborhood point cloud is lower than a preset value. Point count threshold; S33, if the current number of inner points of the target surface weld neighborhood point cloud is not less than the preset point count threshold, the target surface weld neighborhood point cloud is removed from the multi-view fusion weld neighborhood point cloud to obtain the remaining weld point cloud, and the remaining weld point cloud is updated to the multi-view fusion weld neighborhood point cloud and enters step S31; S34, repeat steps S31 to S33 until the current number of inner points of the target surface weld neighborhood point cloud is less than the preset point count threshold, and all target surface weld neighborhood point clouds in the multi-view fusion weld neighborhood point cloud are obtained; wherein, after determining the target surface weld neighborhood point cloud, the target surface weld neighborhood point cloud is plane optimized to obtain the target weld neighborhood plane; S40, determine the corresponding welding weld straight line based on each pair of intersecting target weld neighborhood planes.

[0024] Through research, it is found that in the existing technology, in order to improve the intelligence level of welding robots, various sensors such as inductive sensors, ultrasonic sensors, sound sensors, arc sensors and visual sensors are widely used in robot welding. Among them, visual sensors have been widely used in intelligent welding due to their advantages of non-contact measurement, rich information and high detection accuracy. At present, commonly used three-dimensional visual sensors include stereo vision sensors, RGB-D sensors, laser structured light sensors and coded structured light sensors. However, the stereo vision method often faces problems such as difficulty in matching feature points and low detection accuracy in welding environments with complex backgrounds or sparse textures; the positioning accuracy of RGB-D sensors is poor, which affects the accuracy of welding path generation; laser structured light sensors usually capture workpiece point clouds through manual guidance, and extract weld feature points from them to realize automatic generation of welding paths. The manual operation method reduces the efficiency of path generation and increases the time and labor costs in the work process; coded structured light sensors can quickly obtain point cloud data of welding workpieces and further generate welding paths through point cloud processing technology. However, the existing technology of using coded structured light sensors to generate welding paths has the following three technical problems: it is very difficult to extract welds in complex three-dimensional space; point cloud interference outside the workpiece may affect the extraction results; the current weld extraction work is carried out under a single perspective, which is difficult to meet the welding accuracy requirements in complex application scenarios.

[0025] The weld seam recognition and path planning method for a three-dimensional steel structure workpiece provided by the present invention first establishes a multi-perspective fusion point cloud corresponding to a target workpiece; then uses a weld seam neighborhood point cloud segmentation model to perform weld seam segmentation on the multi-perspective fusion point cloud to obtain a multi-perspective fusion weld seam neighborhood point cloud, and the weld seam neighborhood segmentation model can extract the weld seam area for independent analysis to avoid the influence of complex background and other environmental factors; then the multi-perspective fusion weld seam neighborhood point cloud is plane-fitted and the corresponding weld seam straight line is determined based on each pair of intersecting target weld seam neighborhood planes; the plane fitting method proposed by the present invention has faster fitting speed and higher accuracy, and can avoid interference between adjacent planes; the weld seam recognition and path planning method for a three-dimensional steel structure workpiece provided by the present invention can determine the welding path and welding posture without the CAD model of the target weldment (target workpiece), can be applicable to the full-process intelligent welding of any complex three-dimensional workpiece composed of multiple planes, and shows good applicability under complex backgrounds, has the advantages of automatic efficiency, high recognition accuracy and strong adaptability, and meets the intelligent and flexible operation requirements of welding robots.

[0026] It can be understood that step S31 mainly includes: processing the current multi-view fusion weld neighborhood point cloud to obtain multiple single-plane weld neighborhood point clouds, and determining the single-plane weld neighborhood point cloud with the largest number of internal points as the target surface weld neighborhood point cloud.

[0027] It can be understood that in the scheme of the present invention, the target workpiece is a three-dimensional steel structure, and the target workpiece (complex three-dimensional workpiece) can be a workpiece composed of a base plate and three vertical plates, wherein the vertical plate is vertically arranged on the base plate, and the two relatively parallel vertical plates are connected at the end face through another vertical plate.

[0028] Furthermore, step S10 specifically includes: S11, using a coded structured light sensor to obtain a target structure point cloud data set of the target workpiece under multiple perspectives, wherein one perspective corresponds to a structure point cloud data set; S12, converting the local point cloud in each structure point cloud data set to the robot base coordinate system through a rotation matrix, wherein the rotation matrix is ​​a transformation matrix that maps the sensor coordinate system to the robot base coordinate system; S13, combining the k-means clustering algorithm and the iterative closest point algorithm to align the local point clouds after coordinate conversion, and obtain a multi-perspective fused point cloud formed by the fusion of all local point clouds.

[0029] In the specific implementation, the coded structured light sensor is driven by the robot arm to move to different positions to obtain the target structure point cloud data set of the target workpiece under multiple perspectives. The coded structured light sensor has the function of obtaining the point cloud of the object surface, and its measurement accuracy is less than 1mm. The frame rate of obtaining the depth map (the number of frames of the depth map that the depth sensor or depth camera can generate per unit time) is greater than 1 frame per second; the multi-degree-of-freedom robot (welding robot) is used to carry the welding gun and the sensor on the hand (coded structured light sensor), respond to instructions to execute the shooting movement of the sensor on the hand and the welding movement of the welding gun, and use the hand-eye calibration results and the posture of the center point of the multi-degree-of-freedom robot end effector relative to the robot base coordinate system to calculate the rotation matrix from the sensor coordinate system to the robot base coordinate system.

[0030] In the specific implementation, the converted point cloud is registered by combining the k-means clustering algorithm and the iterative closest point algorithm. Among them, the k-means clustering algorithm reduces the computational cost by reducing the search range of the iterative closest point algorithm. The details are as follows: Assume that the point cloud Gather into k centroids of clusters, where , k is usually 2 to 10. For each point , first according to the point The distance between each cluster centroid determines the point distance The algorithm then compares each point by performing a naive search With point The distance d and find the minimum distance , determine the departure point nearest point Appended to the nearest point In the list, as a point The corresponding point of .

[0031] Furthermore, the steps of establishing the weld neighborhood point cloud segmentation model include: S201, obtaining a workpiece point cloud training data set corresponding to the training workpiece; S202, using a data enhancement method to expand the number of workpiece point cloud training data sets, and using a principal component analysis method to calculate the normal vector information of each training point cloud data in the expanded workpiece point cloud training data set; S203, annotating the workpiece weld region of interest (workpiece weld neighborhood) in the expanded workpiece point cloud training data set with weld labels, and establishing a weld neighborhood point cloud segmentation model by training the workpiece point cloud training data set with weld labels containing normal vector information, wherein the width of the workpiece weld region of interest is within a width threshold range.

[0032] Optionally, the width threshold is 40 mm; in order to ensure the diversity of weld images, the point cloud types in the workpiece point cloud training data set are enriched by changing the posture of the sensor and welding; data enhancement methods include rotation transformation, random jitter, random translation, scale transformation, random sampling, random deletion and other methods.

[0033] Furthermore, the weld neighborhood point cloud segmentation model includes a feature extraction module, a cross-attention module, and multiple fully connected layers. The feature extraction module is a U-shaped structure based on PointNet and PointNet++. The feature extraction module uses set abstraction and feature propagation to predict the point cloud semantic features of the training point cloud data. ; The semantic features of point clouds are transformed through the cross attention module Input into the shared multi-layer perceptron to obtain the key feature map , query feature graph and value feature graph ; Using the formula Calculate the output feature map , where i∈M, Softmax is a normalized exponential function; , and It is the addition, subtraction and multiplication of matrices. Through the fully connected layer, a mapping relationship between the output feature map and the training point cloud data annotated with the weld label is established to establish a weld neighborhood point cloud segmentation model to achieve classification or regression of point cloud features. During the model training process, the cross entropy loss function is used to evaluate the difference between the model prediction value and the actual annotation value to optimize the model.

[0034] Optionally, the expression of the cross entropy loss function is , where n is the number of categories, is the actual marked value, is the model's predicted value.

[0035] Furthermore, in step S30, the multi-view fusion weld neighborhood point cloud obtained after segmentation is subjected to multi-plane fitting (single plane segmentation one by one) using an improved random sampling consistency algorithm, which mainly includes three steps: single plane fitting, multi-plane fitting and plane optimization.

[0036] Furthermore, the step of obtaining a single single-plane weld neighborhood point cloud includes: randomly selecting 3 sampling points from the multi-view fused weld neighborhood point cloud, calculating the plane normal vector of the sampling plane corresponding to the 3 sampling points, wherein the distance between the sampling points is less than a preset sampling threshold; calculating the plane parameters of the sampling plane and determining the plane equation; calculating the distance from all points to the sampling plane, and using the error function to evaluate and determine the inliers, wherein the points whose distances are less than the preset distance threshold are regarded as inliers: repeating the above steps until the number of inliers and / or the number of iterations meet the confidence condition.

[0037] Further, the step of plane optimization includes: calculating the The centroid of the target surface weld neighborhood point cloud is obtained, and the centroid neighborhood point cloud is queried by establishing a KD tree; two sampling points and the centroid are randomly selected on the extracted centroid neighborhood point cloud for plane fitting to obtain the target weld neighborhood plane.

[0038] In the specific implementation, three sampling points are randomly selected from the multi-view fusion weld neighborhood point cloud. , , Calculate the plane normal vector , where the distance between the sampling points is less than the preset sampling threshold D, ; Using formula Calculate the plane parameter d of the sampling plane; Determine the plane equation to be expressed as ; Calculate each point The distance to the sampling plane and using the error function Evaluation to determine the interior points; In the above formula, , the distance is less than the preset point threshold T 1 The points are considered as inliers. Repeat the above steps until the number of inliers and the number of iterations meet the confidence condition. ,in, , I is the number of current internal points, N is the total number of points in the point cloud, k represents the current number of iterations, is the desired confidence threshold, representing the desired probability of success.

[0039] In specific implementation, the inner points corresponding to the planes that have been historically fitted are eliminated and the above steps are repeated from the remaining weld point clouds until the inner point ratio is lower than the preset point count threshold T2, and the iteration is stopped.

[0040] In the specific implementation, first, calculate the The centroid of the target surface weld neighborhood point cloud is then queried by establishing a KD tree. The centroid calculation expression is: ; Secondly, randomly select two points on the extracted centroid neighborhood point cloud ( and ) to perform plane fitting and obtain the optimized plane parameters, whose expression is: .

[0041] Furthermore, step S40 also includes identifying the welding path and welding posture according to the welding seam straight line.

[0042] Preferably, step S40 specifically includes: S41, determining the weld intersection line and weld centerline vector formed by the intersection of each pair of intersecting target weld neighborhood planes; S42, extracting the point cloud in the neighborhood of the weld intersection line by a clustering algorithm, and projecting all the point clouds inside the cluster onto the weld intersection line; S43, sorting the projection point set obtained by the projection, and determining the starting projection point and the ending projection point; S44, sampling the weld segment between the starting projection point and the ending projection point with a fixed step size to obtain a discrete point set ; S45, establish a discrete coordinate system relative to the robot base coordinate system at each discrete point, wherein the x-axis of the discrete coordinate system is determined by the weld direction vector, the z-axis is determined by the weld centerline vector, and the y-axis is determined by the right-hand rule; use the direction vectors of the x-axis, y-axis and z-axis to construct a rotation matrix to determine the welding path and welding posture of the multi-degree-of-freedom robot.

[0043] In the specific implementation, the analytical formula of the weld intersection line and the center line vector extraction are determined. For each pair of intersecting planes (Plane 1) and (Plane 2), its normal vector and Respectively expressed as , ; Direction vector of weld intersection line The formula can be obtain; A point on the weld intersection line can be solved by solving the linear equation system , and set z = 0 to calculate: The linear analytical expression of the intersection line can be determined by a point on the weld intersection line and the direction vector. By formula , , to confirm.

[0044] In the specific implementation, the point cloud in the neighborhood of the weld line (weld intersection line) is extracted through the clustering algorithm, and then all the points inside the cluster are projected onto the weld line. The projected point set is sorted to determine the starting projection point and the ending projection point in the straight line direction. The two projection endpoints represent the starting and ending positions of the weld. The weld line segment between the starting and ending points is sampled with a fixed step size to obtain a discrete point set. A discrete coordinate system relative to the robot base coordinate system is established at each discrete point. The x-axis is determined by the weld direction vector, the z-axis is determined by the weld centerline vector, and the y-axis is determined by the right-hand rule. The rotation matrix is ​​constructed using the direction vectors of the x, y, and z axes to determine the Euler angle of the multi-degree-of-freedom robot tool end.

[0045] The weld seam identification and path planning method for a three-dimensional steel structure workpiece provided by the present invention has the following beneficial effects: the improved point cloud splicing and registration technology can improve the splicing accuracy of multi-view workpiece point clouds, thereby improving the extraction accuracy of weld seams; the weld seam neighborhood point cloud segmentation model can extract the weld seam area for independent analysis, which can avoid the influence of complex background and other environmental factors; the point cloud multi-plane fitting technology is faster and more accurate than the traditional fitting technology, and the centroid-based fitting optimization can avoid the influence between adjacent planes; the welding trajectory planning method composed of the welding path and the welding posture can quickly convert the extracted weld seam parameters into the robot's motion parameters, thereby guiding the robot to weld; without the need for a CAD model of the welding workpiece, it can be applied to the full-process intelligent welding of any complex three-dimensional workpiece, and shows good applicability under complex backgrounds. The intelligent welding system has the advantages of automatic efficiency, high recognition accuracy and strong adaptability, meeting the flexible and intelligent operation requirements of welding robots.

[0046] The present invention also provides a steel structure three-dimensional workpiece weld seam recognition and path planning system, a welding robot and a coded structured light sensor, the welding robot has a base and a manipulator, the coded structured light sensor is arranged on the manipulator, and a processing device is arranged on the welding robot, and the processing device is used to implement the steps of the above-mentioned steel structure three-dimensional workpiece weld seam recognition and path planning method.

[0047] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned steel structure three-dimensional workpiece weld identification and path planning method are implemented.

[0048] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for weld seam recognition and path planning of a three-dimensional steel structure workpiece, characterized in that: The steps include: S10, based on multiple structural point cloud data sets, point cloud stitching and registration technology is used to establish a multi-view fusion point cloud corresponding to the target workpiece; S20, performing weld segmentation on the multi-view fused point cloud using a weld neighborhood point cloud segmentation model to obtain a multi-view fused weld neighborhood point cloud; S30, performing point cloud plane fitting on the multi-view fused weld neighborhood point cloud to obtain multiple target weld neighborhood planes, specifically including: S31, processing the current multi-view fused weld neighborhood point cloud to obtain a single-plane weld neighborhood point cloud with the largest number of internal points as the target surface weld neighborhood point cloud; S32, determining whether the current number of inner points of the target surface weld neighborhood point cloud is lower than a preset point number threshold; S33, if the current number of inner points of the target surface weld neighborhood point cloud is not less than the preset point number threshold, the target surface weld neighborhood point cloud is removed from the multi-view fused weld neighborhood point cloud to obtain the remaining weld point cloud, the remaining weld point cloud is updated to the multi-view fused weld neighborhood point cloud and the process proceeds to step S31; S34, repeating steps S31 to S33 until the current number of inner points of the target surface weld neighborhood point cloud is lower than the preset point number threshold, and obtaining all the target surface weld neighborhood point clouds in the multi-view fused weld neighborhood point cloud; After determining the target surface weld neighborhood point cloud, plane optimization is performed on the target surface weld neighborhood point cloud to obtain a target weld neighborhood plane; S40, determining a corresponding welding seam straight line based on each pair of intersecting target weld seam neighborhood planes.

2. The method for weld seam recognition and path planning of a three-dimensional steel structure workpiece according to claim 1, characterized in that: The steps of obtaining a single point cloud of the single-plane weld neighborhood include: Randomly select three sampling points from the multi-view fusion weld neighborhood point cloud, and calculate the plane normal vectors of the sampling plane formed by the three sampling points, where the distance between the sampling points is less than a preset sampling threshold; Calculating the plane parameters of the sampling plane and determining the plane equation; Calculate the distance from all points to the sampling plane and use the error function to evaluate and determine the inliers, where points with distances less than the preset distance threshold are considered inliers: Repeat the above steps until the number of inliers and / or the number of iterations meet the confidence condition.

3. The method for weld seam recognition and path planning of a three-dimensional steel structure workpiece according to claim 1, characterized in that: The steps for plane optimization include: Calculate the The centroid of the target surface weld neighborhood point cloud is obtained by establishing a KD tree to query the centroid neighborhood point cloud; Two sampling points and the centroid are randomly selected on the extracted centroid neighborhood point cloud for plane fitting to obtain a target weld neighborhood plane.

4. The method for weld seam recognition and path planning of a three-dimensional steel structure workpiece according to any one of claims 1 to 3, characterized in that: The steps to establish the weld neighborhood point cloud segmentation model include: S201, obtaining a workpiece point cloud training data set corresponding to the training workpiece; S202, using a data enhancement method to expand the number of workpiece point cloud training data sets, and using a principal component analysis method to calculate normal vector information of each training point cloud data in the expanded workpiece point cloud training data set; S203, annotating the workpiece weld region of interest in the expanded workpiece point cloud training data set with a weld label, and establishing the weld neighborhood point cloud segmentation model by training the workpiece point cloud training data set annotated with weld labels containing normal vector information, wherein the width of the workpiece weld region of interest is within a width threshold range.

5. The method for weld seam recognition and path planning of a three-dimensional steel structure workpiece according to claim 4, characterized in that: The weld neighborhood point cloud segmentation model includes a feature extraction module, a cross-attention module, and multiple fully connected layers. The feature extraction module is a U-shaped structure based on PointNet and PointNet++. The feature extraction module uses set abstraction and feature propagation to predict the point cloud semantic features of the training point cloud data. ; The point cloud semantic features are transformed into Input into the shared multi-layer perceptron to obtain the key feature map , query feature graph and value feature graph ; Using formula Calculate the output feature map , where i∈M, Softmax is a normalized exponential function; , and is matrix addition, subtraction, and multiplication; The mapping relationship between the output feature map and the training point cloud data annotated with the weld label is established through the fully connected layer to establish the weld neighborhood point cloud segmentation model, so as to realize the classification or regression of the point cloud features; During the model training process, the cross entropy loss function is used to evaluate the difference between the model prediction value and the actual labeled value to optimize the model.

6. The method for weld seam recognition and path planning of a three-dimensional steel structure workpiece according to any one of claims 1 to 3, characterized in that: Step S40 also includes identifying a welding path and a welding posture according to the welding seam straight line; Step S40 specifically includes: S41, determining a weld intersection line and a weld centerline vector formed by the intersection of each pair of intersecting target weld neighborhood planes; S42, extracting point clouds in the neighborhood of the weld intersection line by a clustering algorithm, and projecting all point clouds in the cluster onto the weld intersection line; S43, sorting the projection point set obtained by projection, and determining the starting projection point and the ending projection point; S44, the weld segment between the starting projection point and the ending projection point is sampled with a fixed step size to obtain a discrete point set ; S45, establishing a discrete coordinate system relative to the robot base coordinate system at each discrete point, wherein the x-axis of the discrete coordinate system is determined by the weld direction vector, the z-axis is determined by the weld centerline vector, and the y-axis is determined by the right-hand rule; using the direction vectors of the x-axis, y-axis and z-axis to construct a rotation matrix to determine the welding path and welding posture of the multi-degree-of-freedom robot.

7. The method for weld seam recognition and path planning of a three-dimensional steel structure workpiece according to claim 6, characterized in that: Using formula Determine the weld centerline vector ,in, , , , are the normal vectors of a pair of intersecting target weld neighborhood planes, and d is the direction vector of the weld intersection line.

8. The method for weld seam recognition and path planning of a three-dimensional steel structure workpiece according to any one of claims 1 to 3, characterized in that: Step S10 specifically includes: S11, using a coded structured light sensor to obtain a target structure point cloud dataset of the target workpiece under multiple viewing angles, wherein one viewing angle corresponds to one structure point cloud dataset; S12, converting each local point cloud in the structure point cloud data set to a robot base coordinate system through a rotation matrix, wherein the rotation matrix is ​​a conversion matrix that maps the sensor coordinate system to the robot base coordinate system; S13, combining the k-means clustering algorithm and the iterative closest point algorithm to align the local point clouds after the coordinate transformation, and obtain a multi-view fused point cloud formed by fusion of all the local point clouds.

9. A steel structure three-dimensional workpiece weld identification and path planning system, characterized by: It includes a welding robot and a coded structured light sensor, wherein the welding robot has a base and a robot arm, and the coded structured light sensor is arranged on the robot arm; A processing device is arranged on the welding robot, and the processing device is used to implement the steps of the steel structure three-dimensional workpiece weld recognition and path planning method as described in any one of claims 1 to 8.

Images