Welding robot, welding track planning method and device thereof, storage medium and computer program product

Abstract

The invention discloses a welding track planning method and device of a welding robot, the welding robot, a storage medium and a computer program product, and the method comprises the following steps: inputting point cloud data of a workpiece into a point cloud semantic segmentation model to obtain welding seam point cloud data of a linear welding seam of the workpiece; fitting the welding seam point cloud data to obtain a welding seam welding spot of the linear welding seam; welding spot fine adjustment and pose estimation are carried out on the welding spot of the welding seam, and the welding track of the linear welding seam is obtained. According to the scheme, the semantic segmentation model is used for processing the point cloud data, welding seam fitting and welding spot fine adjustment are carried out, the point cloud of the linear welding seam can be accurately recognized and extracted, the efficiency and accuracy of welding seam detection are improved, and the precision and adaptability of welding track planning are further improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of welding robots, and particularly relates to a welding trajectory planning method, device, welding robot, storage medium, and computer program product for a welding robot, and more particularly to a welding trajectory planning method, device, welding robot, storage medium, and computer program product for a straight weld seam based on three-dimensional point cloud. Background Art

[0002] As one of the most common and critical welding forms, the accuracy of the welding trajectory planning for a straight weld seam directly affects the welding quality. With the development of automation technology, robotic welding technology has gradually replaced manual welding, significantly improving the welding efficiency and consistency. However, robotic welding systems usually rely on preset welding trajectories and are difficult to adapt to geometric deviations of workpieces or changes in weld seam positions, resulting in unstable welding quality.

[0003] The above content is only used to assist in understanding the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The purpose of the present invention is to provide a welding trajectory planning method, device, welding robot, storage medium, and computer program product for a welding robot, so as to solve the problem in the related solutions that robotic welding systems usually rely on preset welding trajectories and are difficult to adapt to geometric deviations of workpieces or changes in weld seam positions, resulting in unstable welding quality of straight weld seams, and achieve the effect of not only accurately identifying and extracting the point cloud of straight weld seams, improving the efficiency and accuracy of weld seam detection, but also improving the accuracy and adaptability of welding trajectory planning by using a semantic segmentation model to process point cloud data and performing weld seam fitting and solder joint fine-tuning.

[0005] The present invention provides a welding trajectory planning method for a welding robot, including: obtaining point cloud data of a workpiece; inputting the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain weld seam point cloud data of a straight weld seam of the workpiece; fitting the weld seam point cloud data to obtain weld points of the straight weld seam; and performing solder joint fine-tuning and pose estimation on the weld points to obtain a welding trajectory of the straight weld seam.

[0006] In some embodiments, the point cloud semantic segmentation model includes: a topological feature learning module, a geometric feature learning module, a feature fusion module, a label prediction module, and a point cloud extraction module; inputting the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain the weld point cloud data of the straight weld of the workpiece, including: inputting the point cloud data of the workpiece into the topological feature learning module to obtain global topological features; inputting the point cloud data of the workpiece into the geometric feature learning module to obtain global geometric features; inputting the global topological features and the global geometric features into the feature fusion module to obtain fused features; using the label prediction module to process the fused features to obtain the probability that each point cloud data in the point cloud data of the workpiece belongs to the point cloud data of the straight weld; and the point cloud extraction module extracting the weld point cloud data of the complete straight weld according to the probability that each point cloud data belongs to the point cloud data of the straight weld.

[0007] In some embodiments, fitting the weld point cloud data to obtain the weld solder joints of the straight weld, including: using density clustering with normal vector constraints to segment the weld point cloud data to obtain a weld point cloud set; the weld point cloud set includes point cloud clusters; performing straight line fitting on each point cloud cluster in the weld point cloud set to obtain the fitting straight line of each point cloud cluster; projecting all points in each point cloud cluster onto the corresponding fitting straight line, and uniformly sampling coordinate points at a preset interval within the projection range to generate the weld solder joints of the straight weld.

[0008] In some embodiments, using density clustering with normal vector constraints to segment the weld point cloud data to obtain a weld point cloud set, including: determining whether the point cloud clusters in the weld point cloud data meet a preset first condition and a preset second condition; if the point cloud cluster meets the preset first condition and the preset second condition, adding the point cloud cluster to the weld point cloud set; if the point cloud cluster does not meet the preset first condition or the preset second condition, not adding the point cloud cluster to the weld point cloud set; wherein, the preset first condition is that the number of point cloud data within a preset first range of the point cloud cluster is greater than a point cloud data quantity threshold; the preset second condition is that the included angle of the normal vectors of all point cloud data within a preset first range of the point cloud cluster is less than or equal to an angle threshold.

[0009] In some embodiments, performing solder joint fine-tuning and pose estimation on the weld solder joints to obtain the welding trajectory of the straight weld includes: for each solder joint in the weld solder joints, denoting the set of point cloud data of all solder joints within a preset second range of the current solder joint as the local point cloud set; determining whether it is possible to segment from the local point cloud set a number of planes greater than or equal to the plane number threshold; if it is not possible to segment from the local point cloud set a number of planes greater than or equal to the plane number threshold, then not performing solder joint fine-tuning on the current solder joint; if it is possible to segment from the local point cloud set a number of planes greater than or equal to the plane number threshold, then calculating the intersection line of the segmented planes; projecting the current solder joint onto the intersection line to obtain a projection point; using the projection point as the solder joint after solder joint fine-tuning of the current solder joint; performing pose estimation on all solder joints that have completed solder joint fine-tuning to obtain the welding trajectory of the straight weld.

[0010] In some embodiments, performing pose estimation on all solder joints that have completed solder joint fine-tuning to obtain the welding trajectory of the straight weld further includes: for each solder joint that has completed solder joint fine-tuning, normalizing the direction vector of the intersection line of the current solder joint to obtain the x-axis basis vector; in the base coordinate system of the robotic arm, taking the point on one side of the weld of the workpiece as the viewpoint, calculating the centroid of the set of points where the neighborhood point cloud of the current solder joint is projected onto the intersection line to obtain the direction vector from the viewpoint to the centroid, denoted as the first direction vector; calculating the set of normal vectors of the plane, and determining whether there is a normal vector in the set of normal vectors whose angle with the first direction vector is an obtuse angle; if there is a normal vector whose angle with the first direction vector is an obtuse angle, then taking the opposite of the direction of the normal vector whose angle with the first direction vector is an obtuse angle; if there is no normal vector whose angle with the first direction vector is an obtuse angle, then summing all the normal vectors in the set of normal vectors to obtain the second direction vector; calculating the component of the second direction vector perpendicular to the x-axis basis vector, and normalizing it to obtain the z-axis basis vector; determining the y-axis basis vector according to the x-axis basis vector and the z-axis basis vector; taking the coordinates of each solder joint in the weld solder joints as the origin of the coordinate system, and the x-axis basis vector, the z-axis basis vector, and the y-axis basis vector form the coordinate system orientation to form the welding trajectory of the straight weld.

[0011] Matched with the above method, on the other hand, the present invention provides a welding trajectory planning device for a welding robot, including: an acquisition unit configured to acquire the point cloud data of a workpiece; a semantic segmentation unit configured to input the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain the weld point cloud data of the straight weld of the workpiece; a fitting unit configured to fit the weld point cloud data to obtain the weld solder joints of the straight weld; a trajectory planning unit configured to perform solder joint fine-tuning and pose estimation on the weld solder joints to obtain the welding trajectory of the straight weld.

[0012] In some embodiments, the point cloud semantic segmentation model includes: a topological feature learning module, a geometric feature learning module, a feature fusion module, a label prediction module, and a point cloud extraction module; a semantic segmentation unit that inputs the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain the weld point cloud data of the linear weld of the workpiece, including: inputting the point cloud data of the workpiece into the topological feature learning module to obtain global topological features; inputting the point cloud data of the workpiece into the geometric feature learning module to obtain global geometric features; inputting the global topological features and the global geometric features into the feature fusion module to obtain fused features; using the label prediction module to process the fused features to obtain the probability that each point cloud data in the point cloud data of the workpiece belongs to the point cloud data of the linear weld; and the point cloud extraction module extracts the weld point cloud data of the complete linear weld according to the probability that each point cloud data belongs to the point cloud data of the linear weld.

[0013] In some embodiments, a fitting unit fits the weld point cloud data to obtain the weld solder joints of the linear weld, including: using density clustering with normal vector constraints to segment the weld point cloud data to obtain a weld point cloud set; the weld point cloud set includes point cloud clusters; performing linear fitting on each point cloud cluster in the weld point cloud set to obtain the fitting line of each point cloud cluster; projecting all points in each point cloud cluster onto the corresponding fitting line, and uniformly sampling coordinate points at a preset interval within the projection range to generate the weld solder joints of the linear weld.

[0014] In some embodiments, the fitting unit uses density clustering with normal vector constraints to segment the weld point cloud data to obtain a weld point cloud set, including: determining whether the point cloud clusters in the weld point cloud data meet a preset first condition and a preset second condition; if the point cloud cluster meets the preset first condition and the preset second condition, adding the point cloud cluster to the weld point cloud set; if the point cloud cluster does not meet the preset first condition or the preset second condition, not adding the point cloud cluster to the weld point cloud set; wherein, the preset first condition is that the number of point cloud data within a preset first range of the point cloud cluster is greater than a point cloud data quantity threshold; the preset second condition is that the included angle of the normal vectors of all point cloud data within the preset first range of the point cloud cluster is less than or equal to an angle threshold.

[0015] In some embodiments, the trajectory planning unit performs solder joint fine-tuning and pose estimation on the weld solder joints to obtain the welding trajectory of the straight weld, including: for each solder joint in the weld solder joints, denoting the set of point cloud data of all solder joints within a preset second range of the current solder joint as the local point cloud set; determining whether it is possible to segment from the local point cloud set a number of planes greater than or equal to the plane quantity threshold; if it is not possible to segment from the local point cloud set a number of planes greater than or equal to the plane quantity threshold, then no solder joint fine-tuning is performed on the current solder joint; if it is possible to segment from the local point cloud set a number of planes greater than or equal to the plane quantity threshold, then calculating the intersection line of the segmented planes; projecting the current solder joint onto the intersection line to obtain a projection point; taking the projection point as the solder joint after solder joint fine-tuning of the current solder joint; performing pose estimation on all solder joints that have completed solder joint fine-tuning to obtain the welding trajectory of the straight weld.

[0016] In some embodiments, the trajectory planning unit performs pose estimation on all solder joints that have completed solder joint fine-tuning to obtain the welding trajectory of the straight weld, and further includes: for each solder joint that has completed solder joint fine-tuning, normalizing the direction vector of the intersection line of the current solder joint to obtain the x-axis basis vector; in the base coordinate system of the robotic arm, taking the point on the weld side of the workpiece as the viewing point, calculating the centroid of the point set obtained by projecting the neighborhood point cloud of the current solder joint onto the intersection line, to obtain the direction vector from the viewing point to the centroid, denoted as the first direction vector; calculating the set of normal vectors of the plane, and determining whether there is a normal vector in the set of normal vectors whose included angle with the first direction vector is an obtuse angle; if there is a normal vector whose included angle with the first direction vector is an obtuse angle, then taking the opposite of the direction of the normal vector whose included angle with the first direction vector is an obtuse angle; if there is no normal vector whose included angle with the first direction vector is an obtuse angle, then summing all the normal vectors in the set of normal vectors to obtain the second direction vector; calculating the component of the second direction vector perpendicular to the x-axis basis vector, and normalizing it to obtain the z-axis basis vector; determining the y-axis basis vector according to the x-axis basis vector and the z-axis basis vector; taking the coordinates of each solder joint in the weld solder joints as the origin of the coordinate system, and the x-axis basis vector, the z-axis basis vector, and the y-axis basis vector form the coordinate system orientation, to form the welding trajectory of the straight weld.

[0017] Matched with the above device, on the other hand, the present invention provides a welding robot, including: the welding trajectory planning device of the above-mentioned welding robot.

[0018] Matched with the above method, on the other hand, the present invention provides a storage medium, the storage medium includes a stored program, wherein, when the program runs, it controls the device where the storage medium is located to execute the welding trajectory planning method of the above-mentioned welding robot.

[0019] Matching the above method, on the other hand, the present invention provides a computer program product, which includes a computer program. When the computer program product is processed and executed, the steps of the welding trajectory planning method of the above welding robot are realized.

[0020] In the solution of the present invention, the point cloud data of the workpiece is input into the point cloud semantic segmentation model to obtain the weld point cloud data of the straight weld of the workpiece; the weld point cloud data is fitted to obtain the weld points of the straight weld; the weld points are fine-tuned and pose estimated to obtain the welding trajectory of the straight weld. Therefore, not only can the point cloud of the straight weld be accurately identified and extracted, improving the efficiency and accuracy of weld detection, but also the accuracy and adaptability of welding trajectory planning can be improved.

[0021] Other features and advantages of the present invention will be described in the following specification, and part of them will be obvious from the specification or understood by implementing the present invention.

[0022] The technical solution of the present invention will be further described in detail below through the drawings and embodiments. Description of the Drawings

[0023] Figure 1 It is a schematic flowchart of an embodiment of the welding trajectory planning method of the welding robot of the present invention; Figure 2 It is a schematic structural diagram of an embodiment of the welding trajectory planning device of the welding robot of the present invention; Figure 3 It is the architecture diagram of the weld point cloud semantic segmentation model SPSNet; Figure 4 It is a schematic diagram of the topological feature learning module in the weld point cloud semantic segmentation model; Figure 5 It is a schematic diagram of the geometric feature learning module in the weld point cloud semantic segmentation model; Figure 6 It is the flowchart of weld fitting; Figure 7 It is the flowchart of weld point fine-tuning and pose estimation.

[0024] Combined with the drawings, the reference numerals in the embodiments of the present invention are as follows: 101 - Acquisition unit; 102 - Semantic segmentation unit; 103 - Fitting unit; 104 - Trajectory planning unit. Detailed Embodiments

[0025] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0026] In modern manufacturing, welding technology is an important process to ensure product quality and safety, and it has wide applications especially in fields such as automobile manufacturing, aerospace, and shipbuilding. The premise of welding trajectory planning is to obtain the position information of the weld seam through weld seam detection. Traditional straight weld seam detection methods mainly rely on manual detection or contact sensor technology. However, manual detection is inefficient and easily affected by subjective factors, making it difficult to ensure the accuracy and consistency of detection results. Although the detection method based on contact sensors can provide certain automation support, there are problems such as insufficient detection accuracy and poor adaptability. And robotic welding is difficult to adapt to geometric deviations of workpieces or changes in weld seam positions, resulting in unstable welding quality.

[0027] Therefore, the present invention provides a welding trajectory planning method for a welding robot. By obtaining high-precision point cloud data of the workpiece to be welded, and then based on deep learning technology, a point cloud semantic segmentation model SPSNet (Seam Point Cloud Segmentation Network) is constructed to learn the geometric and topological features of the weld seam, enabling accurate recognition and efficient extraction of the point cloud of straight weld seams in different types of welding workpieces in complex environments. At the same time, based on the extracted point cloud of straight weld seams, through weld seam fitting and local weld point fine-tuning, the dynamic adjustment and optimization of the welding trajectory are realized, effectively adapting to geometric deviations of workpieces and changes in weld seam positions, and improving the accuracy of the welding trajectory.

[0028] According to an embodiment of the present invention, there is provided a welding trajectory planning method for a welding robot, as Figure 1 shown in the flowchart of an embodiment of the method of the present invention. The welding trajectory planning method for the welding robot may include: step S110 to step S140.

[0029] At step S110, obtain the point cloud data of the workpiece.

[0030] The three-dimensional coordinate point set on the surface of the workpiece can be collected by a 3D camera to form point cloud data including weld seam and non-weld seam areas, which serves as the basic input for subsequent processing.

[0031] At step S120, input the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain the weld seam point cloud data of the straight weld seam of the workpiece.

[0032] The point cloud semantic segmentation model is the SPSNet model. The SPSNet model is used to fuse the topological features and geometric features of the weld seam, distinguish the weld seam from the non-weld seam point cloud through deep learning, accurately identify the weld seam point cloud data of the straight weld seam, solve the problems of large subjective errors in manual detection and poor adaptability of contact sensors, and achieve the automatic and high-precision extraction of the weld seam position.

[0033] As Figure 3 shown, the point cloud semantic segmentation model includes: a topological feature learning module, a geometric feature learning module, a feature fusion module, a label prediction module, and a point cloud extraction module.

[0034] The topological feature learning module is used to extract the topological features of the local connection relationship of the point cloud, which characterizes the local structural information of the weld seam. Its principle is to use 2 layers of EdgeConv (graph neural network convolution) to construct a point cloud neighborhood graph (knn graph), calculate the edge features between points, map the edge features to high-dimensional features through MLP, and output the global topological feature f top , with a dimension of 1×512.

[0035] The geometric feature learning module is used to extract the global morphological features of the point cloud, which characterizes the overall trend of the weld seam. Its principle is to use 4 layers of MLP (64, 64, 128, 512) to map the original point cloud coordinates to high-dimensional features, compress the feature dimension through max pooling, and output the global geometric feature f geo , with a dimension of 1×512.

[0036] The feature fusion module is used to splice the topological and geometric features to form a complete representation containing local and global information. Specifically, it splices f top and f geo to obtain a concatenated feature of 1×1024, maps and replicates the feature through MLP, and outputs the fused feature f fus , with a dimension of n×1024 (n is the number of points in the point cloud).

[0037] The label prediction module is used to convert the fused feature into the class probability of the point cloud to distinguish the weld seam from the non-weld seam points. The principle is to reduce the dimension through 4 fully connected layers (512, 256, 128, 64), and finally map it to a 2D feature through a fully connected layer (the number of classes c = 2), and apply the softmax function to obtain the probability that each point belongs to the weld seam.

[0038] The point cloud extraction module filters the weld seam point cloud based on the probability and outputs a pure straight weld seam coordinate set. Specifically, a probability threshold (such as 0.5) is set, the points with probabilities higher than the threshold are retained, and their three-dimensional coordinates are extracted.

[0039] The pre-training process of the point cloud semantic segmentation model is as follows: Use a 3D camera to collect point cloud data of a certain number of different types of workpieces to be welded, manually annotate the semantic labels of the point cloud, and divide the workpiece point cloud into two categories: weld point cloud and other point cloud. Then, combined with point cloud data augmentation methods such as random rotation, scaling, translation, and jittering, the number of training samples is increased and the diversity of training samples is improved to produce a model training and test data set. Among them, the weld point cloud includes weld point clouds of various welding methods, such as straight welds, curved welds, circular welds, etc. Point cloud data augmentation can increase the number of training samples and the diversity of training samples, thereby improving the generalization ability and segmentation accuracy of the model.

[0040] In some embodiments, in step S120, the specific process of inputting the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain the weld point cloud data of the straight weld of the workpiece includes: steps S210 to S250.

[0041] Step S210, input the point cloud data of the workpiece into the topological feature learning module to obtain global topological features.

[0042] As Figure 3 shown, the point cloud data of the workpiece will be input into the topological feature learning module and the geometric feature learning module respectively. The topological feature reflects the connection relationship between points and can effectively represent the local structure information of the weld. The logic of the topological feature learning module is as Figure 4 shown. The topological feature learning module consists of 2 EdgeConv. EdgeConv is a convolutional operation in a graph neural network. The first EdgeConv extracts weld topological features of size (n×64) from the input point cloud of size (n×3). For the input point cloud, first construct a k-nearest neighbor graph structure (knn graph) to obtain point cloud edge features of size (n×k×6), and then use a multi-layer perceptron (mlp) composed of 2 fully connected layers to map the point cloud edge features into high-dimensional features of size (n×k×64). After max pooling, point cloud topological features f of size (n×64) are obtained. edge1 .

[0043] Assume x i , i∈n is a point in the input point cloud, and x i , i∈k is a point in the k nearest neighbors of x i , then the edge feature hθ(x i , x j ) between x i and x j is expressed as follows: hθ(x i , x j ) = hθ(x i , x i-x j ); where θ is a learnable parameter.

[0044] The second EdgeConv then maps the input f edge1 further to the point cloud topological feature f of (n×128). edge2 Compared with f edge1 , f edge2 has a larger receptive field. To obtain multi-scale weld structure information, f edge1 and f edge2 are concatenated, and a multi-layer perceptron (mlp) is used to map them into high-dimensional features of size (n×512). Finally, the weld topological feature f of size (1×512) is obtained through max pooling. top .

[0045] Step S220: Input the point cloud data of the workpiece into the geometric feature learning module to obtain the global geometric feature.

[0046] The weld topological feature f top can represent the local structure information of the weld, but lacks the global structure information of the weld. Therefore, a weld geometric feature learning module is constructed to extract the global geometric feature of the weld. The logic of the geometric feature learning module is as Figure 5 shown. For the input point cloud of size (n×3), 4 multi-layer perceptrons (mlp) are used to map the point cloud into high-dimensional features of size (n×512), and then the weld geometric feature f of size (1×512) is finally obtained through max pooling. geo , representing the global geometric structure information of the weld.

[0047] Step S230: Input the global topological feature and the global geometric feature into the feature fusion module to obtain the fusion feature.

[0048] In the feature fusion module, the extracted weld topological feature f top and the weld geometric feature f geo are further concatenated to obtain a concatenated feature of size (1×1024). Then, a multi-layer perceptron is used to map the feature into a feature of size (1×1024), and this feature is copied n times to obtain the fusion feature f of size (n×1024). fus . f fus contains both the local structure information and the global structure information of the weld and can identify the weld more accurately.

[0049] Step S240: Use the label prediction module to process the fusion feature to obtain the probability that each point cloud data in the point cloud data of the workpiece belongs to the point cloud data of the straight weld.

[0050] Input the fused feature f fus into the label prediction module to learn the label information of the point cloud. In the label prediction module, a multi-layer perceptron composed of 4 fully connected layers is used to map the feature dimensions of the fused feature f fus to 512, 256, 128, and 64 dimensions in sequence, obtaining a feature of size (n×64). Finally, a fully connected layer is used to map the feature dimension to c, where c represents the category of the point cloud and can be set to 2, obtaining a feature of size (n×2). Apply the softmax function to this feature to convert each output value into a probability between 0 and 1, obtaining the label category information of the point cloud.

[0051] Step S250: The point cloud extraction module extracts the weld point cloud data of the complete straight weld according to the probability that each point cloud data belongs to the point cloud data of the straight weld.

[0052] In the point cloud extraction module, the weld point cloud is extracted according to the point cloud category probability. Through the fusion of topological features and geometric features, the straight weld in complex workpieces can be accurately identified. The label prediction and point cloud extraction replace manual operations, shortening the weld detection time and improving the detection efficiency.

[0053] At step S130, the weld point cloud data is fitted to obtain the weld solder joints of the straight weld.

[0054] The fitting process can filter out noise points, adapt to the uneven surface of the workpiece or point cloud noise, avoid the preset trajectory mismatch caused by the geometric deviation of the workpiece, and make the solder joints fit the actual weld trend.

[0055] In some embodiments, in step S130, the specific process of fitting the weld point cloud data to obtain the weld solder joints of the straight weld includes: steps S310 to S330.

[0056] Step S310: Use density clustering with normal vector constraints to segment the weld point cloud data to obtain a weld point cloud set; the weld point cloud set includes point cloud clusters.

[0057] In some embodiments, in step S310, the specific process of segmenting the weld seam point cloud data by using density clustering with normal vector constraints to obtain a weld seam point cloud set includes: determining whether the point cloud clusters in the weld seam point cloud data meet a preset first condition and a preset second condition; if a point cloud cluster meets the preset first condition and the preset second condition, adding the point cloud cluster to the weld seam point cloud set; if a point cloud cluster does not meet the preset first condition or the preset second condition, not adding the point cloud cluster to the weld seam point cloud set; wherein, the preset first condition is that the number of point cloud data within a preset first range of the point cloud cluster is greater than a point cloud data quantity threshold; the preset second condition is that the included angle of the normal vectors of all point cloud data within the preset first range of the point cloud cluster is less than or equal to an angle threshold.

[0058] The point cloud of a straight weld seam should present a high-density aggregation feature in a local area, while noise points or non-weld points are usually sparsely distributed. Isolated noise points can be filtered out through density screening, and the real weld seam area can be retained. For example, the number of random noise points on the workpiece surface within a neighborhood of 0.01 m is usually less than 10 and will be excluded. The surface normal vectors of a straight weld seam should generally point in the same direction, while the normal vector directions in the non-weld areas of complex workpieces vary greatly. By constraining the included angle of the normal vectors, it can be ensured that the point cloud cluster conforms to the geometric features of the straight weld seam and non-straight structures are excluded.

[0059] Specifically, the weld seam point cloud is segmented by using point cloud density clustering with normal vector constraints to obtain independent straight weld seam point clouds. Among them, point cloud density clustering with normal vector constraints is a clustering method that simultaneously considers point cloud density information and normal vector direction information. The density information represents the local density of the point cloud by the number of points within a radius neighborhood. The more the number of points, the higher the density. The density of the points in the same point cloud cluster, that is, the number of points within a radius neighborhood, should be greater than the point number threshold. For example, the number of points within a radius of 0.01 m is greater than 10. The normal vector directions of the points in the same point cloud cluster generally point in the same direction. For example, if the included angle between the normal vectors of points does not exceed 20 degrees, it is considered that the normal vector directions generally point in the same direction. When the number of points in the point cloud cluster obtained after segmentation is greater than 50, it is considered that the point cloud cluster is a valid weld seam point cloud and is retained to obtain independent straight weld seam point clouds S = {s1, …, s t}, where t represents the number of straight weld seam point clouds.

[0060] In complex welded workpieces, the point clouds of different weld seam segments are connected to each other and the point cloud densities of different weld seam segments are similar. Relying only on point cloud density information cannot segment these weld seam point clouds into independent weld seam segment point clouds. Since the normal vectors of different weld seam segment point clouds vary greatly, by adding normal vector constraint conditions and simultaneously considering point cloud density information and normal vector information, the segmentation of the connected weld seam segment point clouds can be realized to obtain independent weld seam segment point clouds. At the same time, points with a density lower than the threshold are regarded as noise points and removed, which can also play a role in filtering noise points.

[0061] By identifying valid weld point cloud clusters based on density conditions and normal vector conditions, the extraction accuracy of straight welds in complex workpieces is improved, and the reliability of the segmentation results is ensured.

[0062] Step S320: Perform straight line fitting on each point cloud cluster in the weld point cloud set to obtain the fitted straight line of each point cloud cluster.

[0063] Straight line fitting is performed on each point cloud cluster using the Random Sample Consensus (RANSAC) algorithm. Specifically, for the weld point cloud s i , i ∈ (1, t)n in the weld point cloud set S, the RANSAC algorithm is used to perform straight line fitting on this point cloud to obtain the straight line equation. Through the inlier screening mechanism of the RANSAC algorithm, it dynamically adapts to the local bending or deformation of the weld, making the fitted straight line closer to the actual weld trend.

[0064] Step S330: Project all points in each point cloud cluster onto the corresponding fitted straight line, and uniformly sample coordinate points at a preset interval within the projection range to generate the weld points of the straight weld.

[0065] Project the weld point cloud s i onto this straight line, and sample coordinate points at equal intervals within the point cloud projection range to obtain the weld point set P = {p1, …, p m} of this weld point cloud, where m represents the number of weld points. By converting the continuous straight line into discrete weld points executable by the robot, the trajectory accuracy and execution efficiency are balanced.

[0066] Figure 6 It is a flowchart of weld fitting. As Figure 6 shown, this method includes: Step 1: Segment the weld point cloud using point cloud density clustering with normal vector constraints to obtain independent straight weld point clouds.

[0067] Step 2: Determine whether the point cloud meets the conditions of density exceeding the threshold and consistent normal vector direction. If it meets, execute Step 3; if not, discard this point cloud.

[0068] Step 3: For the obtained straight weld point cloud S = {s1, …, s t}, use the RANSAC algorithm to perform straight line fitting on this point cloud to obtain the straight line equation. Project the weld point cloud s i onto this straight line, and sample coordinate points at equal intervals within the point cloud projection range to obtain the weld point set P = {p1, …, p m}.

[0069] At step S140, the weld spots are finely adjusted and the pose is estimated to obtain the welding trajectory of the straight weld seam.

[0070] The fine adjustment of the weld spots can solve the global fitting error, adapt to the local deformation of the workpiece, avoid the instability of the welding quality caused by the geometric deviation of the workpiece, and ensure that the weld spots accurately fall on the actual weld seam. The pose estimation is used to endow the weld spots with orientation information, so that the welding torch pose adapts to the normal direction of the weld seam surface, avoid the welding defects caused by the fixed welding torch angle in the preset trajectory, and improve the consistency of the welding quality.

[0071] By using 3D point cloud and SPSNet model to replace manual annotation and contact sensors, the accuracy and adaptability problems in the detection link are solved, providing a reliable data basis for trajectory planning. The intelligent conversion from point cloud to trajectory and pose estimation improve the accuracy of trajectory planning and the welding adaptability of complex workpieces.

[0072] In some embodiments, in step S140, the specific process of finely adjusting the weld spots and estimating the pose of the weld spots to obtain the welding trajectory of the straight weld seam includes: step S410 to step S450.

[0073] Step S410, for each weld spot in the weld spots, the set of point cloud data of all weld spots within a preset second range of the current weld spot is recorded as the local point cloud set.

[0074] The global point cloud may contain distant noise points or non-weld structures, and directly processing it is likely to cause fine adjustment deviation. Therefore, fine adjustment of the weld spots is required. First, for each weld spot, all point cloud data is searched within a preset second range (such as a radius of 0.02 m) centered on it to form a local point cloud set. Narrowing the processing range in this way can improve the calculation efficiency, reduce unnecessary point cloud interference, and accurately capture the true weld seam morphology near the weld spots.

[0075] Step S420, determine whether it is possible to segment a plane with a number greater than or equal to the plane number threshold from the local point cloud set.

[0076] Step S430, if it is not possible to segment a plane with a number greater than or equal to the plane number threshold from the local point cloud set, then no fine adjustment of the current weld spot is performed.

[0077] The straight weld seam can be approximated as the intersection line of two planes locally. If enough planes cannot be segmented, it means that the local point cloud may be noise or non-weld structure. The random sample consensus algorithm is used to segment the plane from the local point cloud set, and it is determined whether a plane with a number ≥ the plane number threshold (usually 2) can be segmented.

[0078] Step S440: If it is possible to segment a number of planes greater than or equal to the plane number threshold from the local point cloud set, calculate the intersection lines of the segmented planes; project the current solder joint onto the intersection lines to obtain projection points; use the projection points as the solder joints after fine-tuning the solder joints.

[0079] If ≥2 planes are segmented, calculate the plane intersection line, project the solder joint onto the intersection line to obtain the projection point, and use it as the position of the solder joint after fine-tuning. In this way, the intersection line of the two planes is the true position of the local weld seam, and the projection operation can correct this deviation, and the solder joint accurately falls on the actual weld seam.

[0080] Specifically, there may be noise points in the weld point cloud s i . The straight line fitted by the random sample consensus algorithm does not completely coincide with the weld seam, resulting in some solder joints not being on the weld seam, and the solder joints need to be fine-tuned. For each solder joint in P, first use radius neighborhood search to search for the local point cloud within a radius of 0.02 m around it, and then use the random sample consensus algorithm to segment two planes that make up the weld seam from the local neighborhood point cloud. If the number of segmented planes is less than 2 (such as the workpiece point cloud is incomplete), the solder joint is not fine-tuned. Otherwise, calculate the intersection line and the plane normal vector of the two planes. Within the local range, the intersection line of the two planes can be regarded as a straight line coinciding with the weld seam. Project the solder joint onto the intersection line to obtain the projection point set R = {r1,…,r k}, calculate the plane normal vector to obtain the plane normal vector set V = {v1,…,v k}, where k represents the number of solder joints after projection. Finally, perform linear fitting on the projection point set R = {r1,…,r k} and project all the solder joints in P = {p1,…,p m} onto this straight line to obtain the fine-tuned solder joints P r ={p r 1,…,p r m}. Through local weld seam fine-tuning, projecting the solder joints in P onto the weld seam can overcome the geometric deviation of the workpiece and the change of the weld seam position, and significantly improve the position accuracy of the solder joints.

[0081] Step S450: Perform pose estimation on all the solder joints that have completed fine-tuning to obtain the welding trajectory of the straight weld seam.

[0082] Pose estimation is to construct a workpiece coordinate system on the solder joint. By analyzing the local point cloud set, judging plane segmentation, and fine-tuning the intersection line projection of the weld seam solder joints, and combining pose estimation to construct a workpiece coordinate system, the position of the solder joint can be accurately corrected and dynamically adapted to the geometric characteristics of the weld seam, effectively overcoming the influence of workpiece geometric deviation and weld seam position change, significantly improving the accuracy and adaptability of welding trajectory planning, and providing reliable technical support for intelligent welding of robots.

[0083] In some embodiments, in step S450, the specific process of performing pose estimation on all solder joints that have completed fine adjustment of the solder joints to obtain the welding trajectory of the linear weld seam includes: for each solder joint that has completed fine adjustment of the complete solder joint, normalizing the direction vector of the intersection line of the current solder joint to obtain the x-axis basis vector; in the robot base coordinate system, taking a point on one side of the weld seam of the workpiece as the viewing point, calculating the centroid of the point set obtained by projecting the neighborhood point cloud of the current solder joint onto the intersection line, and obtaining the direction vector from the viewing point to the centroid, denoted as the first direction vector; calculating the set of normal vectors of the plane, and determining whether there is a normal vector in the set of normal vectors whose included angle with the first direction vector is an obtuse angle; if there is a normal vector whose included angle with the first direction vector is an obtuse angle, then reversing the direction of the normal vector whose included angle with the first direction vector is an obtuse angle; if there is no normal vector whose included angle with the first direction vector is an obtuse angle, then summing all the normal vectors in the set of normal vectors to obtain the second direction vector; calculating the component of the second direction vector perpendicular to the x-axis basis vector, and normalizing it to obtain the z-axis basis vector; determining the y-axis basis vector according to the x-axis basis vector and the z-axis basis vector; taking the coordinates of each solder joint in the weld seam solder joints as the origin of the coordinate system, and using the x-axis basis vector, the z-axis basis vector, and the y-axis basis vector to form the coordinate system orientation, thereby forming the welding trajectory of the linear weld seam.

[0084] Specifically, first, take the direction vector of the straight line fitted by the projection point set R as the direction vector of the x-axis of the solder joint workpiece coordinate system, and normalize it to obtain the x-axis basis vector v x . Then, in the robot base coordinate system, select a point in the space inside the workpiece (one side of the weld seam) as the viewing point p v . Calculate the centroid point p k of the projection point set R = {r1,..., r center}, and obtain the direction vector v v from the viewing point p center to p vc . Next, calculate the included angle between the vector v vc and each normal vector in the set of normal vectors V. If it is an acute angle, it is considered that the direction of the normal vector is consistent with the direction of v vc . Otherwise, reverse the direction of the normal vector to obtain the set of plane normal vectors V r = {v r 1,..., v r m} with unified orientation. Then, add the normal vectors in V r to obtain the direction vector v normal , calculate the component of it perpendicular to the x-axis basis vector v x , and normalize it to obtain the basis vector v z of the z-axis of the workpiece coordinate system.Finally, based on the right-hand rule, the y-axis basis vector v of the workpiece coordinate system is calculated y On the same straight weld, the workpiece coordinate system corresponding to the coordinate axes of different welding points have the same orientation, and the coordinate origin is the weld point's own coordinate, thereby obtaining the precise welding trajectory of the weld in the robot base coordinate system.

[0085] The x-axis is determined by normalizing the direction vector of the intersection line of the fine-tuned weld, the z-axis is generated by unifying the normal vector direction by combining the viewpoint and the centroid vector, and the y-axis is determined according to the right-hand system. A workpiece coordinate system with the weld as the origin is constructed. The position and posture of each weld can be accurately defined, and the welding trajectory can be adaptive to the geometric deviation of the workpiece, ensuring that the welding gun welds along the direction of the weld and perpendicular to the weld surface, which significantly improves the accuracy of the welding trajectory and the process adaptability.

[0086] Figure 7 The flowchart of solder joint fine-tuning and pose estimation is as follows: Figure 7 As shown, the method includes: Step 11, for each weld point in the weld, use radius neighbor search to search for the local point cloud within a radius of 0.02m, and then use the random sampling consistency algorithm to segment the plane that makes up the weld from the local neighborhood point cloud.

[0087] Step 12, determine whether the number of the segmented planes is less than 2. If the number is less than 2, skip the welding point, otherwise execute step 13.

[0088] Step 13, calculate the intersection line of the two planes and the plane normal vectors, and use the sum of the plane normal vectors as the z-axis vector of the welding point.

[0089] Step 14, project the solder joint onto the plane intersection line to obtain the projection point, which is the solder joint after fine-tuning.

[0090] Step 15, project the weld point onto the straight line fitted by the projection point, and use the direction vector of the straight line as the x-axis vector of the weld.

[0091] Step 16, calculate the z-axis vector of the weld and obtain the position and posture of each weld point based on the x-axis vector to obtain the weld trajectory.

[0092] Through weld fitting and local fine-tuning technology of weld points, dynamic adjustment and optimization of welding trajectory can be achieved, the accuracy of welding trajectory can be improved, and the problem that the preset welding trajectory of the robot is difficult to adapt to the geometric deviation of the workpiece and the change of weld position can be solved, providing technical guarantee for the widespread application of robot welding technology.

[0093] Adopting the technical solution of this embodiment, the point cloud data of the workpiece is input into the point cloud semantic segmentation model to obtain the weld point cloud data of the straight weld of the workpiece; the weld point cloud data is fitted to obtain the weld points of the straight weld; the weld points are finely adjusted and the pose is estimated to obtain the welding trajectory of the straight weld. Thus, it can not only accurately identify and extract the point cloud of the straight weld, improve the efficiency and accuracy of weld detection, but also improve the accuracy and adaptability of welding trajectory planning.

[0094] According to an embodiment of the present invention, there is also provided a welding robot welding trajectory planning device corresponding to the welding trajectory planning method of the welding robot. Refer to Figure 2 The structural schematic diagram of an embodiment of the device of the present invention shown in the figure. The welding robot welding trajectory planning device may include: an acquisition unit 101, a semantic segmentation unit 102, a fitting unit 103, and a trajectory planning unit 104.

[0095] The acquisition unit 101 is configured to acquire the point cloud data of the workpiece.

[0096] The three-dimensional coordinate point set on the surface of the workpiece can be collected by a 3D camera to form the point cloud data including the weld and non-weld areas, which is used as the basic input for subsequent processing.

[0097] The semantic segmentation unit 102 is configured to input the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain the weld point cloud data of the straight weld of the workpiece.

[0098] The point cloud semantic segmentation model is the SPSNet model. The SPSNet model is used to fuse the topological features and geometric features of the weld, distinguish the weld and non-weld point clouds through deep learning, accurately identify the weld point cloud data of the straight weld, solve the problems of large subjective error in manual detection and poor adaptability of contact sensors, and realize the automatic high-precision extraction of the weld position.

[0099] As Figure 3 shown, the point cloud semantic segmentation model includes: a topological feature learning module, a geometric feature learning module, a feature fusion module, a label prediction module, and a point cloud extraction module.

[0100] The topological feature learning module is used to extract the topological features of the local connection relationship of the point cloud, representing the local structure information of the weld. Its principle is to use 2 layers of EdgeConv (graph neural network convolution) to construct a point cloud neighborhood graph (knn graph), calculate the edge features between points, map the edge features to high-dimensional features through MLP, and output the global topological feature f top , with a dimension of 1×512.

[0101] The geometric feature learning module is used to extract the global morphological features of the point cloud, representing the overall trend of the weld seam. Its principle is to map the original point cloud coordinates into high-dimensional features using a 4-layer MLP (64, 64, 128, 512), compress the feature dimension through max pooling, and output the global geometric feature f geo , with a dimension of 1×512.

[0102] The feature fusion module is used to splice the topological and geometric features to form a complete representation containing local and global information. Specifically, it splices f top and f geo to obtain a spliced feature of 1×1024, maps and replicates the feature through an MLP, and outputs the fused feature f fus , with a dimension of n×1024 (n is the number of points in the point cloud).

[0103] The label prediction module is used to convert the fused feature into the class probability of the point cloud to distinguish between weld seam and non-weld seam points. The principle is to reduce the dimension through 4 fully connected layers (512, 256, 128, 64), and finally map it into a 2D feature through a fully connected layer (the number of classes c = 2), and apply the softmax function to obtain the probability that each point belongs to the weld seam.

[0104] The point cloud extraction module then filters the weld seam point cloud based on the probability and outputs a pure set of linear weld seam coordinates. Specifically, a probability threshold (such as 0.5) is set, points with a probability higher than the threshold are retained, and their three-dimensional coordinates are extracted.

[0105] The pre-training process of the point cloud semantic segmentation model is as follows: using a 3D camera to collect point cloud data of a certain number of different types of workpieces to be welded, manually annotating the semantic labels of the point cloud, classifying the workpiece point cloud into weld seam point cloud and other point cloud, and then combining point cloud data augmentation methods such as random rotation, scaling, translation, and jittering to increase the number of training samples and improve the diversity of training samples, and making model training and test data sets. Among them, the weld seam point cloud includes weld seam point clouds of various welding methods, such as linear weld seams, curved weld seams, circular weld seams, etc. Point cloud data augmentation can increase the number of training samples and the diversity of training samples, thereby improving the generalization ability and segmentation accuracy of the model.

[0106] In some embodiments, the semantic segmentation unit 102 inputs the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain the weld seam point cloud data of the linear weld seam of the workpiece, including: The semantic segmentation unit 102 is specifically further configured to input the point cloud data of the workpiece into the topological feature learning module to obtain the global topological feature.

[0107] Such as Figure 3As shown, the point cloud data of the workpiece is respectively input into the topological feature learning module and the geometric feature learning module. The topological feature reflects the connection relationship between points and can effectively represent the local structural information of the weld seam. The logic of the topological feature learning module is as Figure 4 shown. The topological feature learning module consists of 2 EdgeConv, and EdgeConv is a convolutional operation in a graph neural network. The first EdgeConv extracts the weld seam topological feature of size (n×64) from the input point cloud of size (n×3). For the input point cloud, first construct a k-nearest neighbor graph structure (knn graph) to obtain the point cloud edge feature of size (n×k×6), and then use a multi-layer perceptron (mlp) composed of 2 fully connected layers to map the point cloud edge feature into a high-dimensional feature of size (n×k×64). After max pooling, the point cloud topological feature f edge1 is obtained.

[0108] Assume x i , i∈n is a point in the input point cloud, and x i , i∈k is a point among the k nearest neighbors of x i . Then the edge feature hθ(x i , x j ) between x i and x j is expressed as follows: hθ(x i , x j ) = hθ(x i , x i - x j ); where θ is a learnable parameter.

[0109] The second EdgeConv further maps the input f edge1 to the point cloud topological feature f edge2 of size (n×128). Compared with f edge1 , f edge2 has a larger receptive field. In order to obtain multi-scale weld seam structure information, f edge1 and f edge2 are concatenated, and a multi-layer perceptron (mlp) is used to map them into a high-dimensional feature of size (n×512). Finally, through max pooling, the weld seam topological feature f top of size (1×512) is obtained.

[0110] The semantic segmentation unit 102 is specifically further configured to input the point cloud data of the workpiece into the geometric feature learning module to obtain the global geometric feature.

[0111] The weld seam topological feature f topIt can characterize the local structural information of the weld seam, but lacks the global structural information of the weld seam. Therefore, a weld seam geometric feature learning module is constructed to extract the global geometric features of the weld seam. The logic of the geometric feature learning module is as Figure 5 shown. For the input point cloud with a size of (n×3), four multi-layer perceptrons (MLPs) are used to map the point cloud into a high-dimensional feature of (n×512), and then through max pooling, the weld seam geometric feature f geo with a size of (1×512) is finally obtained, which characterizes the global geometric structural information of the weld seam.

[0112] The semantic segmentation unit 102 is specifically further configured to input the global topological feature and the global geometric feature into the feature fusion module to obtain a fused feature.

[0113] In the feature fusion module, the extracted weld seam topological feature f top and the weld seam geometric feature f geo are further concatenated to obtain a concatenated feature with a size of (1×1024). Then, a multi-layer perceptron is used to map the feature into a feature with a size of (1×1024), and this feature is copied n times to obtain a fused feature f fus with a size of (n×1024). f fus contains both the local structural information and the global structural information of the weld seam, and can identify the weld seam more accurately.

[0114] The semantic segmentation unit 102 is specifically further configured to use the label prediction module to process the fused feature to obtain the probability that each point cloud data in the point cloud data of the workpiece belongs to the point cloud data of the straight weld seam.

[0115] The fused feature f fus is input into the label prediction module to learn the label information of the point cloud. In the label prediction module, a multi-layer perceptron composed of 4 fully connected layers is used to sequentially map the feature dimensions of the fused feature f fus to 512, 256, 128, and 64 dimensions to obtain a feature with a size of (n×64). Finally, a fully connected layer is used to map the feature dimension to c, where c represents the category of the point cloud and can be set to 2 to obtain a feature with a size of (n×2). The softmax function is applied to this feature to convert each output value into a probability between 0 and 1 to obtain the label category information of the point cloud.

[0116] The semantic segmentation unit 102 is specifically further configured to extract the weld seam point cloud data of the complete straight weld seam according to the probability that each point cloud data belongs to the point cloud data of the straight weld seam by the point cloud extraction module.

[0117] In the point cloud extraction module, the weld point cloud is extracted according to the point cloud category probability. By fusing topological features and geometric features, the straight welds in complex workpieces can be accurately identified. And label prediction and point cloud extraction replace manual operations, shortening the weld detection time and improving the detection efficiency.

[0118] The fitting unit 103 is configured to fit the weld point cloud data to obtain the weld points of the straight weld.

[0119] The fitting process can filter out noise points, adapt to the uneven surface of the workpiece or point cloud noise, avoid the mismatch of the preset trajectory caused by the geometric deviation of the workpiece, and make the weld points fit the actual weld direction.

[0120] In some embodiments, the fitting unit 103 fits the weld point cloud data to obtain the weld points of the straight weld, including: The fitting unit 103 is further specifically configured to segment the weld point cloud data by density clustering with normal vector constraints to obtain a weld point cloud set; the weld point cloud set includes point cloud clusters.

[0121] In some embodiments, the fitting unit 103 segments the weld point cloud data by density clustering with normal vector constraints to obtain a weld point cloud set, including: determining whether the point cloud clusters in the weld point cloud data meet a preset first condition and a preset second condition; if the point cloud cluster meets the preset first condition and the preset second condition, adding the point cloud cluster to the weld point cloud set; if the point cloud cluster does not meet the preset first condition or the preset second condition, not adding the point cloud cluster to the weld point cloud set; wherein, the preset first condition is that the number of point cloud data within a preset first range of the point cloud cluster is greater than a point cloud data quantity threshold; the preset second condition is that the included angle of the normal vectors of all point cloud data within a preset first range of the point cloud cluster is less than or equal to an angle threshold.

[0122] The point cloud of the straight weld should show a high-density aggregation feature in the local area, while the noise points or non-weld points are usually sparsely distributed. Isolated noise points can be filtered out through density screening, and the real weld area can be retained. For example, the number of random noise points on the surface of the workpiece within a neighborhood of 0.01 m is usually less than 10 and will be excluded. The surface normal vectors of the straight welds should generally point in the same direction, while the normal vector directions in the non-weld areas of complex workpieces vary greatly. By constraining the included angle of the normal vectors, it can be ensured that the point cloud clusters conform to the geometric features of the straight welds and non-straight structures can be excluded.

[0123] Specifically, the weld point cloud is segmented using point cloud density clustering with normal vector constraints to obtain independent linear weld point clouds. Among them, point cloud density clustering with normal vector constraints is a clustering method that simultaneously considers point cloud density information and normal vector direction information. The density information represents the local density of the point cloud by the number of points within the radius neighborhood. The more points there are, the higher the density. The density of the points in the same point cloud cluster, that is, the number of points within the radius neighborhood, should be greater than the point number threshold. For example, the number of points within a radius of 0.01 m is greater than 10. The normal vector directions of the points in the same point cloud cluster generally point in the same direction. For example, if the angle between the normal vectors of points does not exceed 20 degrees, it is considered that the normal vector directions generally point in the same direction. For the point cloud clusters obtained after segmentation, when the number of their points is greater than 50, it is considered that the point cloud cluster is a valid weld point cloud and is retained, obtaining independent linear weld point clouds S = {s1, …, s t}, where t represents the number of linear weld point clouds.

[0124] In complex welded workpieces, the point clouds of different weld segments are connected to each other and the point cloud densities of different weld segments are similar. Relying solely on point cloud density information cannot segment these weld point clouds into independent weld segment point clouds. Since the normal vectors of the point clouds of different weld segments vary greatly, by adding normal vector constraint conditions and simultaneously considering point cloud density information and normal vector information, the segmentation of the connected weld segment point clouds can be achieved, obtaining independent weld segment point clouds. At the same time, the points with a density lower than the threshold are regarded as noise points and removed, which can also play a role in filtering noise points.

[0125] By identifying valid weld point cloud clusters according to density conditions and normal vector conditions, the extraction accuracy of linear welds in complex workpieces is improved, ensuring the reliability of the segmentation results.

[0126] The fitting unit 103 is specifically further configured to perform linear fitting on each point cloud cluster in the weld point cloud set to obtain the fitting line of each point cloud cluster.

[0127] Linear fitting is to fit a line to each point cloud cluster using the Random Sample Consensus algorithm (RANSAC). Specifically, for the weld point cloud s i in the weld point cloud set S, i ∈ (1, t)n, the Random Sample Consensus algorithm is used to perform linear fitting on this point cloud to obtain the line equation. Through the inlier screening mechanism of the Random Sample Consensus algorithm, it dynamically adapts to the local bending or deformation of the weld, making the fitting line closer to the actual weld trend.

[0128] The fitting unit 103 is specifically further configured to project all the points in each point cloud cluster onto the corresponding fitting line and uniformly sample coordinate points at a preset interval within the projection range to generate the weld points of the linear weld.

[0129] Project the weld point cloud s iProject onto this straight line, equally spaced sample coordinate points within the point cloud projection range, and obtain the set of welding points P = {p1, …, p m}, where m represents the number of welding points. By converting the continuous straight line into discrete welding points executable by the robot, the trajectory accuracy and execution efficiency are balanced.

[0130] Figure 6 It is a flowchart of weld fitting. As Figure 6 shown, this method includes: Step 1, segment the weld point cloud using point cloud density clustering with normal vector constraints to obtain independent straight weld point clouds.

[0131] Step 2, determine whether the point cloud meets the conditions of density exceeding the threshold and consistent normal vector direction. If it meets the conditions, execute Step 3; if not, discard the point cloud.

[0132] Step 3, for the obtained straight weld point cloud S = {s1, …, s t}, use the random sample consensus algorithm to fit a straight line to this point cloud to obtain the straight line equation. Project the weld point cloud s i onto this straight line, equally spaced sample coordinate points within the point cloud projection range, and obtain the set of welding points P = {p1, …, p m} of this weld point cloud.

[0133] The trajectory planning unit 104 is configured to perform fine-tuning of welding points and pose estimation on the weld points to obtain the welding trajectory of the straight weld.

[0134] The fine-tuning of welding points can solve the global fitting error, adapt to local deformation of the workpiece, avoid unstable welding quality caused by geometric deviation of the workpiece, and ensure that the welding points accurately fall on the actual weld. Pose estimation is used to give the welding point orientation information, so that the welding torch pose adapts to the normal direction of the weld surface, avoid welding defects caused by fixed welding torch angle in the preset trajectory, and improve the consistency of welding quality.

[0135] By using 3D point cloud and SPSNet model to replace manual marking and contact sensors, the accuracy and adaptability problems in the detection link are solved, providing a reliable data basis for trajectory planning. The intelligent conversion from point cloud to trajectory and pose estimation improve the accuracy of trajectory planning and the welding adaptability of complex workpieces.

[0136] In some embodiments, the trajectory planning unit 104 performs fine-tuning of welding points and pose estimation on the weld points to obtain the welding trajectory of the straight weld, including: The trajectory planning unit 104 is specifically further configured to, for each welding point in the weld points, record the set of point cloud data of all welding points within the preset second range of the current welding point as the local point cloud set.

[0137] The global point cloud may contain distant noise points or non-weld structures. Direct processing is likely to cause fine-tuning deviations. Therefore, solder joint fine-tuning is required. First, for each solder joint, search all point cloud data within a preset second range (such as a radius of 0.02 m) centered on it to form a local point cloud set. Narrowing the processing range in this way can improve the calculation efficiency, reduce unnecessary point cloud interference, and accurately capture the true weld form near the solder joint.

[0138] The trajectory planning unit 104 is specifically further configured to determine whether it is possible to segment planes with a quantity greater than or equal to the plane quantity threshold from the local point cloud set.

[0139] The trajectory planning unit 104 is specifically further configured to, if it is not possible to segment planes with a quantity greater than or equal to the plane quantity threshold from the local point cloud set, not perform solder joint fine-tuning on the current solder joint.

[0140] A straight weld can be approximately regarded as the intersection line of two planes locally. If insufficient planes cannot be segmented, it indicates that the local point cloud may be noise or a non-weld structure. Use the random sample consensus algorithm to segment planes from the local point cloud set, and determine whether it is possible to segment planes with a quantity ≥ the plane quantity threshold (usually 2).

[0141] The trajectory planning unit 104 is specifically further configured to, if it is possible to segment planes with a quantity greater than or equal to the plane quantity threshold from the local point cloud set, calculate the intersection line of the segmented planes; project the current solder joint onto the intersection line to obtain a projection point; use the projection point as the solder joint after solder joint fine-tuning of the current solder joint.

[0142] If ≥2 planes are segmented, calculate the plane intersection line, project the solder joint onto the intersection line to obtain a projection point, and use it as the position of the solder joint after fine-tuning. In this way, the intersection line of the two planes is the true position of the local weld, and the projection operation can correct this deviation, causing the solder joint to accurately fall on the actual weld.

[0143] Specifically, there may be noise points in the weld point cloud s i In P, the straight line fitted by the random sample consensus algorithm does not completely coincide with the weld, resulting in some solder joints not being on the weld. Solder joint fine-tuning is required. For each solder joint in P, first use radius neighborhood search to search for the local point cloud within a radius of 0.02 m around it. Then, use the random sample consensus algorithm to segment the two planes that make up the weld from the local neighborhood point cloud. If the number of segmented planes is less than 2 (such as when the workpiece point cloud is incomplete), no fine-tuning is performed on this solder joint. Otherwise, calculate the intersection line of the two planes and the plane normal vectors. Locally, the intersection line of the two planes can be regarded as a straight line coinciding with the weld. Project the solder joint onto the intersection line to obtain the projection point set R = {r1,…,r k}, calculate the plane normal vectors to obtain the plane normal vector set V = {v1, …, v k}, where k represents the number of solder joints after projection. Finally, perform a linear fit on the projected point set R = {r1, …, r k}, and project all the solder joints in P = {p1, …, p m} onto this line to obtain the fine-tuned solder joints P r = {p r 1, …, p r m}. Through local weld fine-tuning, projecting the solder joints in P onto the weld can overcome the geometric deviation of the workpiece and the change in weld position, and significantly improve the position accuracy of the solder joints.

[0144] The trajectory planning unit 104 is further configured to perform pose estimation on all the solder joints that have completed solder joint fine-tuning to obtain the welding trajectory of the straight weld.

[0145] Pose estimation is to construct a workpiece coordinate system on the solder joints. By performing local point cloud set analysis, plane segmentation judgment, and intersection line projection fine-tuning on the weld solder joints, and combining pose estimation to construct a workpiece coordinate system, the position of the solder joints can be accurately corrected and dynamically adapted to the geometric characteristics of the weld, effectively overcoming the influence of workpiece geometric deviation and weld position change, significantly improving the accuracy and adaptability of welding trajectory planning, and providing reliable technical support for intelligent welding of robots.

[0146] In some embodiments, the trajectory planning unit 104 performs pose estimation on all the solder joints that have completed solder joint fine-tuning to obtain the welding trajectory of the straight weld, including: for each solder joint of the complete solder joint fine-tuning, normalizing the direction vector of the intersection line of the current solder joint to obtain the x-axis basis vector; in the base coordinate system of the robotic arm, taking the point on one side of the weld of the workpiece as the viewing point, calculating the centroid of the point set obtained by projecting the neighborhood point cloud of the current solder joint onto the intersection line, and obtaining the direction vector from the viewing point to the centroid, denoted as the first direction vector; calculating the set of plane normal vectors, and determining whether there is a normal vector in the set of normal vectors whose included angle with the first direction vector is an obtuse angle; if there is a normal vector whose included angle with the first direction vector is an obtuse angle, then reversing the direction of the normal vector whose included angle with the first direction vector is an obtuse angle; if there is no normal vector whose included angle with the first direction vector is an obtuse angle, then summing all the normal vectors in the set of normal vectors to obtain the second direction vector; calculating the component of the second direction vector perpendicular to the x-axis basis vector, and normalizing it to obtain the z-axis basis vector; determining the y-axis basis vector according to the x-axis basis vector and the z-axis basis vector; taking the coordinates of each solder joint in the weld solder joints as the origin of the coordinate system, and the x-axis basis vector, the z-axis basis vector, and the y-axis basis vector form the coordinate system orientation to form the welding trajectory of the straight weld.

[0147] Specifically, the direction vector of the straight line fitted by the projection point set R is first used as the direction vector of the x-axis of the weld workpiece coordinate system, and the x-axis basis vector v is obtained after normalization. x Then, in the robot base coordinate system, select a point in the space inside the workpiece (on the weld side) as the viewpoint p v . Calculate the projection point set R={r1,…,r k}The centroid point p center , obtained from the viewpoint p v Point to p center The direction vector v vc . Then calculate the vector v vc The angle between the normal vector and each normal vector in the normal vector set V is an acute angle, and the normal vector direction is considered to be at the same angle as v. vc The direction is consistent, otherwise the direction of the normal vector is reversed to obtain a set of plane normal vectors V with a uniform direction. r ={v r 1,…,v r m}. Then V r Add the normal vectors in to get the direction vector v normal , calculate its basis vector v perpendicular to the x-axis x The components of the workpiece coordinate system are normalized to obtain the basis vector v of the z-axis. z Finally, based on the right-hand rule, the y-axis basis vector v of the workpiece coordinate system is calculated y On the same straight weld, the workpiece coordinate system corresponding to the coordinate axes of different welding points have the same orientation, and the coordinate origin is the weld point's own coordinate, thereby obtaining the precise welding trajectory of the weld in the robot base coordinate system.

[0148] The x-axis is determined by normalizing the direction vector of the intersection line of the fine-tuned weld, the z-axis is generated by unifying the normal vector direction by combining the viewpoint and the centroid vector, and the y-axis is determined according to the right-hand system. A workpiece coordinate system with the weld as the origin is constructed. The position and posture of each weld can be accurately defined, and the welding trajectory can be adaptive to the geometric deviation of the workpiece, ensuring that the welding gun welds along the direction of the weld and perpendicular to the weld surface, which significantly improves the accuracy of the welding trajectory and the process adaptability.

[0149] Figure 7 The flowchart of solder joint fine-tuning and pose estimation is as follows: Figure 7 As shown, the method includes: Step 11, for each weld point in the weld, use radius neighbor search to search for the local point cloud within a radius of 0.02m, and then use the random sampling consistency algorithm to segment the plane that makes up the weld from the local neighborhood point cloud.

[0150] Step 12, determine whether the number of the segmented planes is less than 2. If the number is less than 2, skip the welding point, otherwise execute step 13.

[0151] Step 13, calculate the intersection line of the two planes and the plane normal vectors, and use the sum of the plane normal vectors as the solder joint z-axis vector.

[0152] Step 14, project the solder joint onto the intersection line of the planes to obtain the projection point, which is the fine-tuned solder joint.

[0153] Step 15, project the solder joint onto the line fitted by the projection point, and use the direction vector of this line as the weld seam x-axis vector.

[0154] Step 16, calculate the weld seam z-axis vector and obtain the pose of each solder joint based on the x-axis vector to obtain the weld seam welding trajectory.

[0155] Through the weld seam fitting and solder joint local fine-tuning technologies, the dynamic adjustment and optimization of the welding trajectory can be realized, the accuracy of the welding trajectory can be improved, the problem that the preset welding trajectory of the robot is difficult to adapt to the geometric deviation of the workpiece and the change of the weld seam position can be solved, and technical support is provided for the wide application of the robot welding technology.

[0156] Since the processing and functions implemented by the device in this embodiment are basically corresponding to the embodiments, principles and examples of the foregoing method, for the parts not detailed in the description of this embodiment, reference can be made to the relevant descriptions in the foregoing embodiments, and details are not repeated here.

[0157] Adopting the technical solution of the present invention, input the point cloud data of the workpiece into the point cloud semantic segmentation model to obtain the weld seam point cloud data of the straight weld seam of the workpiece; fit the weld seam point cloud data to obtain the weld seam solder joints of the straight weld seam; perform solder joint fine-tuning and pose estimation on the weld seam solder joints to obtain the welding trajectory of the straight weld seam. Thus, not only can the point cloud of the straight weld seam be accurately identified and extracted, the efficiency and accuracy of weld seam detection be improved, but also the accuracy and adaptability of welding trajectory planning can be improved.

[0158] According to an embodiment of the present invention, there is also provided a welding robot corresponding to the welding trajectory planning device of the welding robot. The welding robot may include: the welding trajectory planning device of the welding robot described above.

[0159] Since the processing and functions implemented by the welding robot in this embodiment are basically corresponding to the embodiments, principles and examples of the foregoing device, for the parts not detailed in the description of this embodiment, reference can be made to the relevant descriptions in the foregoing embodiments, and details are not repeated here.

[0160] Adopt the technical solution of the present invention, input the point cloud data of the workpiece into the point cloud semantic segmentation model to obtain the weld point cloud data of the straight weld of the workpiece; fit the weld point cloud data to obtain the weld points of the straight weld; perform weld point fine-tuning and pose estimation on the weld points to obtain the welding trajectory of the straight weld. Thus, not only can the point cloud of the straight weld be accurately identified and extracted, improving the efficiency and accuracy of weld detection, but also the accuracy and adaptability of welding trajectory planning can be improved.

[0161] According to an embodiment of the present invention, there is also provided a storage medium corresponding to the welding trajectory planning method of a welding robot. The storage medium includes a stored program, wherein when the program runs, it controls the device where the storage medium is located to execute the above-mentioned welding trajectory planning method of the welding robot.

[0162] Since the processing and functions implemented by the storage medium of this embodiment are basically corresponding to the embodiments, principles and examples of the foregoing method, the details not described in the description of this embodiment can be referred to the relevant descriptions in the foregoing embodiments and will not be elaborated here.

[0163] Adopt the technical solution of the present invention, input the point cloud data of the workpiece into the point cloud semantic segmentation model to obtain the weld point cloud data of the straight weld of the workpiece; fit the weld point cloud data to obtain the weld points of the straight weld; perform weld point fine-tuning and pose estimation on the weld points to obtain the welding trajectory of the straight weld. Thus, not only can the point cloud of the straight weld be accurately identified and extracted, improving the efficiency and accuracy of weld detection, but also the accuracy and adaptability of welding trajectory planning can be improved.

[0164] According to an embodiment of the present invention, there is also provided a computer program product corresponding to the welding trajectory planning method of a welding robot. The computer program product includes a computer program, and when the computer program product is processed and executed, it implements the steps of the above-mentioned welding trajectory planning method of the welding robot.

[0165] Since the processing and functions implemented by the computer program product of this embodiment are basically corresponding to the embodiments, principles and examples of the foregoing method, the details not described in the description of this embodiment can be referred to the relevant descriptions in the foregoing embodiments and will not be elaborated here.

[0166] Adopt the technical solution of the present invention, input the point cloud data of the workpiece into the point cloud semantic segmentation model to obtain the weld point cloud data of the straight weld of the workpiece; fit the weld point cloud data to obtain the weld points of the straight weld; perform weld point fine-tuning and pose estimation on the weld points to obtain the welding trajectory of the straight weld. Thus, not only can the point cloud of the straight weld be accurately identified and extracted, improving the efficiency and accuracy of weld detection, but also the accuracy and adaptability of welding trajectory planning can be improved.

[0167] In summary, it is easily understandable to those skilled in the art that, on the premise of no conflict, the above-mentioned advantageous ways can be freely combined and superimposed.

[0168] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the scope of the claims of the present invention.

Claims

1. A welding trajectory planning method for a welding robot, characterized in that, Including: Obtaining the point cloud data of the workpiece; Inputting the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain the weld point cloud data of the straight weld of the workpiece; The preset point cloud semantic segmentation model is used to determine the weld point cloud data of the straight weld of the workpiece according to the global topological features and global geometric features of the workpiece point cloud data; Fitting the weld point cloud data to obtain the weld points of the straight weld; Performing weld point fine-tuning and pose estimation on the weld points to obtain the welding trajectory of the straight weld.

2. The welding trajectory planning method of the welding robot according to claim 1, characterized in that, The point cloud semantic segmentation model includes: a topological feature learning module, a geometric feature learning module, a feature fusion module, a label prediction module, and a point cloud extraction module; Inputting the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain the weld point cloud data of the straight weld of the workpiece, including: Inputting the point cloud data of the workpiece into the topological feature learning module to obtain global topological features; Inputting the point cloud data of the workpiece into the geometric feature learning module to obtain global geometric features; Inputting the global topological features and the global geometric features into the feature fusion module to obtain fusion features; Using the label prediction module to process the fusion features to obtain the probability that each point cloud data in the point cloud data of the workpiece belongs to the point cloud data of the straight weld; The point cloud extraction module extracts the complete weld point cloud data of the straight weld according to the probability that each point cloud data belongs to the point cloud data of the straight weld.

3. The welding path planning method of the welding robot according to claim 1 or 2, characterized in that, Fitting the weld point cloud data to obtain the weld points of the straight weld, including: Using density clustering with normal vector constraints to segment the weld point cloud data to obtain a weld point cloud set; the weld point cloud set includes point cloud clusters; Performing straight line fitting on each point cloud cluster in the weld point cloud set to obtain the fitting straight line of each point cloud cluster; Projecting all points in each point cloud cluster onto the corresponding fitting straight line, and uniformly sampling coordinate points at a preset interval within the projection range to generate the weld points of the straight weld.

4. The welding trajectory planning method of the welding robot according to claim 3, characterized in that, Using density clustering with normal vector constraints to segment the weld point cloud data to obtain a weld point cloud set, including: Judging whether the point cloud clusters in the weld point cloud data meet a preset first condition and a preset second condition; If the point cloud cluster meets the preset first condition and the preset second condition, adding the point cloud cluster to the weld point cloud set; If the point cloud cluster does not meet the preset first condition or the preset second condition, not adding the point cloud cluster to the weld point cloud set; Wherein, the preset first condition is that the number of point cloud data within a preset first range of the point cloud cluster is greater than the point cloud data quantity threshold; the preset second condition is that the included angle of the normal vectors of all point cloud data within a preset first range of the point cloud cluster is less than or equal to the angle threshold.

5. The welding path planning method of the welding robot according to claim 1, characterized in that Performing weld point fine-tuning and pose estimation on the weld points to obtain the welding trajectory of the straight weld, including: For each weld point in the weld points, denoting the set of point cloud data of all weld points within a preset second range of the current weld point as a local point cloud set; Determine whether planes with a quantity greater than or equal to the plane quantity threshold can be segmented from the local point cloud set; If planes with a quantity greater than or equal to the plane quantity threshold cannot be segmented from the local point cloud set, no fine-tuning of the current welding spot is performed; If planes with a quantity greater than or equal to the plane quantity threshold can be segmented from the local point cloud set, calculate the intersection lines of the segmented planes; project the current welding spot onto the intersection lines to obtain projection points; use the projection points as the welding spots after fine-tuning the current welding spot; Perform pose estimation on all welding spots that have completed fine-tuning to obtain the welding trajectory of the straight weld seam.

6. The welding trajectory planning method of the welding robot according to claim 5, characterized in that, Performing pose estimation on all welding spots with complete fine-tuning to obtain the welding trajectory of the straight weld seam further includes: For each welding spot with complete fine-tuning, normalize the direction vector of the intersection line of the current welding spot to obtain the x-axis basis vector; In the base coordinate system of the robotic arm, use the point on one side of the weld seam of the workpiece as the viewing point, calculate the centroid of the point set obtained by projecting the neighborhood point cloud of the current welding spot onto the intersection line, obtain the direction vector from the viewing point to the centroid, denoted as the first direction vector; Calculate the set of normal vectors of the plane, and determine whether there is a normal vector in the set of normal vectors whose included angle with the first direction vector is an obtuse angle; If there is a normal vector whose included angle with the first direction vector is an obtuse angle, reverse the direction of the normal vector whose included angle with the first direction vector is an obtuse angle; If there is no normal vector whose included angle with the first direction vector is an obtuse angle, sum all the normal vectors in the set of normal vectors to obtain the second direction vector; calculate the component of the second direction vector perpendicular to the x-axis basis vector, and after normalization, obtain the z-axis basis vector; Determine the y-axis basis vector according to the x-axis basis vector and the z-axis basis vector; Take the coordinates of each welding spot in the weld seam as the origin of the coordinate system, and the x-axis basis vector, the z-axis basis vector, and the y-axis basis vector form the coordinate system orientation to form the welding trajectory of the straight weld seam.

7. A welding trajectory planning device for a welding robot, characterized in that, including: An acquisition unit configured to acquire the point cloud data of the workpiece; A semantic segmentation unit configured to input the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain the weld seam point cloud data of the straight weld seam of the workpiece; A fitting unit configured to fit the weld seam point cloud data to obtain the weld spots of the straight weld seam; A trajectory planning unit configured to perform fine-tuning and pose estimation on the weld spots to obtain the welding trajectory of the straight weld seam.

8. A welding robot, characterized in that, including: The welding trajectory planning device of the welding robot according to claim 7.

9. A storage medium, characterized in that, The storage medium includes a stored program, wherein when the program runs, it controls the device where the storage medium is located to execute the welding trajectory planning method of the welding robot according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images