Welding robot and welding trajectory planning method, device, storage medium, and computer program product thereof

Abstract

The present invention discloses a welding trajectory planning method, device, welding robot, storage medium, and computer program product for a welding robot. The method comprises: inputting workpiece point cloud data into a point cloud semantic segmentation model to obtain weld point cloud data of the workpiece's linear weld; fitting the weld point cloud data to obtain weld points of the linear weld; and fine-tuning and estimating the position of the weld points to obtain the welding trajectory of the linear weld. This solution, by processing point cloud data using a semantic segmentation model and performing weld fitting and weld point fine-tuning, not only accurately identifies and extracts point clouds of linear welds, improving the efficiency and accuracy of weld detection, but also enhances the precision and adaptability of welding trajectory planning.

Description

Technical Field

[0001] The present invention belongs to the technical field of welding robots, and specifically relates to a welding trajectory planning method, device, welding robot, storage medium and computer program product of a welding robot, and more particularly to a welding trajectory planning method, device, welding robot, storage medium and computer program product of a linear weld based on a three-dimensional point cloud. Background Art

[0002] As one of the most common and critical welding methods, the accuracy of straight weld trajectory planning directly affects weld quality. With the development of automation technology, robotic welding technology has gradually replaced manual welding, significantly improving welding efficiency and consistency. However, robotic welding systems typically rely on preset welding trajectories and have difficulty adapting to workpiece geometry deviations or changes in weld position, resulting in unstable weld quality.

[0003] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The purpose of the present invention is to provide a welding trajectory planning method, device, welding robot, storage medium and computer program product for a welding robot, so as to solve the problem that the robot welding system in the related scheme usually relies on a preset welding trajectory and is difficult to adapt to the geometric deviation of the workpiece or the change of the weld position, resulting in unstable welding quality of the straight weld. By utilizing the semantic segmentation model to process the point cloud data and perform weld fitting and weld point fine-tuning, it is possible not only to accurately identify and extract the point cloud of the straight weld, improve the efficiency and accuracy of weld detection, but also improve the accuracy and adaptability of welding trajectory planning.

[0005] The present invention provides a welding trajectory planning method for a welding robot, comprising: acquiring point cloud data of a workpiece; inputting the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain weld point cloud data of a linear weld of the workpiece; fitting the weld point cloud data to obtain weld points of the linear weld; and performing weld point fine-tuning and posture estimation on the weld points to obtain a welding trajectory of the linear weld.

[0006] In some embodiments, the point cloud semantic segmentation model includes: a topological feature learning module, a geometric feature learning module, a feature fusion module, a label prediction module, and a point cloud extraction module; the point cloud data of the workpiece is input into a preset point cloud semantic segmentation model to obtain the weld point cloud data of the straight weld of the workpiece, including: inputting the point cloud data of the workpiece into the topological feature learning module to obtain global topological features; inputting the point cloud data of the workpiece into the geometric feature learning module to obtain global geometric features; inputting the global topological features and the global geometric features into the feature fusion module to obtain fusion features; using the label prediction module to process the fusion features to obtain the probability that each point cloud data in the point cloud data of the workpiece belongs to the point cloud data of the straight weld; the point cloud extraction module extracts the complete weld point cloud data of the straight weld according to the probability that each point cloud data belongs to the point cloud data of the straight weld.

[0007] In some embodiments, fitting the weld point cloud data to obtain the weld spots of the straight weld includes: segmenting the weld point cloud data using density clustering with normal vector constraints to obtain a weld point cloud set; the weld point cloud set includes point cloud clusters; performing straight line fitting on each point cloud cluster in the weld point cloud set to obtain a fitted straight line for each point cloud cluster; projecting all points in each point cloud cluster onto the corresponding fitted straight line, and uniformly sampling coordinate points at preset intervals within the projection range to generate weld spots of the straight weld.

[0008] In some embodiments, the weld point cloud data is segmented using density clustering with normal vector constraints to obtain a weld point cloud set, including: determining whether a point cloud cluster in the weld point cloud data satisfies a preset first condition and a preset second condition; if the point cloud cluster satisfies the preset first condition and the preset second condition, the point cloud cluster is added to the weld point cloud set; if the point cloud cluster does not satisfy the preset first condition or the preset second condition, the point cloud cluster is not added to the weld point cloud set; wherein the preset first condition is that the number of point cloud data within a preset first range of the point cloud cluster is greater than a point cloud data number threshold; the preset second condition is that the angle between the normal vectors of all point cloud data within the preset first range of the point cloud cluster is less than or equal to an angle threshold.

[0009] In some embodiments, the weld points of the weld are fine-tuned and the posture is estimated to obtain the welding trajectory of the straight weld, including: for each weld point in the weld, recording the set of point cloud data of all weld points within a preset second range of the current weld point as a local point cloud set; judging whether a number of planes greater than or equal to a plane number threshold can be segmented from the local point cloud set; if a number of planes greater than or equal to a plane number threshold cannot be segmented from the local point cloud set, then no weld fine-tuning is performed on the current weld point; if a number of planes greater than or equal to a plane number threshold can be segmented from the local point cloud set, then calculating the intersection of the segmented planes; projecting the current weld point onto the intersection to obtain a projection point; using the projection point as the weld point after the current weld point is fine-tuned; performing posture estimation on all weld points that have completed weld point fine-tuning to obtain the welding trajectory of the straight weld.

[0010] In some embodiments, the posture estimation is performed on all weld points of the complete weld point fine-tuning to obtain the welding trajectory of the straight weld, and the following further comprises: for each weld point of the complete weld point fine-tuning, the direction vector of the intersection line of the current weld point is normalized to obtain the x-axis basis vector; in the robot arm base coordinate system, the point located on one side of the weld of the workpiece is used as the viewpoint, and the center of mass of the point set of the neighborhood point cloud of the current weld point projected to the intersection line is calculated to obtain the direction vector of the viewpoint pointing to the center of mass, which is recorded as the first direction vector; the normal vector set of the plane is calculated to determine whether there is a normal vector in the normal vector set that has an obtuse angle with the first direction vector; if there is a normal vector with an obtuse angle with the first direction vector If the angle between the direction vector and the normal vector is an obtuse angle, the direction of the normal vector whose angle with the first direction vector is an obtuse angle is reversed; if there is no normal vector whose angle with the first direction vector is an obtuse angle, all normal vectors in the normal vector set are summed to obtain a second direction vector; the component of the second direction vector perpendicular to the x-axis basis vector is calculated, and the z-axis basis vector is obtained after normalization; the y-axis basis vector is determined according to the x-axis basis vector and the z-axis basis vector; the coordinates of each weld point in the weld are used as the origin of the coordinate system, and the x-axis basis vector, the z-axis basis vector, and the y-axis basis vector constitute the coordinate system orientation to form the welding trajectory of the straight weld.

[0011] Matching the above method, the present invention provides a welding trajectory planning device for a welding robot on the other hand, comprising: an acquisition unit, configured to acquire point cloud data of a workpiece; a semantic segmentation unit, configured to input the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain weld point cloud data of the straight weld of the workpiece; a fitting unit, configured to fit the weld point cloud data to obtain weld points of the straight weld; and a trajectory planning unit, configured to perform weld point fine-tuning and pose estimation on the weld points to obtain the welding trajectory of the straight weld.

[0012] In some embodiments, the point cloud semantic segmentation model includes: a topological feature learning module, a geometric feature learning module, a feature fusion module, a label prediction module, and a point cloud extraction module; a semantic segmentation unit inputs the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain weld point cloud data of the straight weld of the workpiece, including: inputting the point cloud data of the workpiece into the topological feature learning module to obtain global topological features; inputting the point cloud data of the workpiece into the geometric feature learning module to obtain global geometric features; inputting the global topological features and the global geometric features into the feature fusion module to obtain fusion features; using the label prediction module to process the fusion features to obtain the probability that each point cloud data in the point cloud data of the workpiece belongs to the point cloud data of the straight weld; the point cloud extraction module extracts the complete weld point cloud data of the straight weld according to the probability that each point cloud data belongs to the point cloud data of the straight weld.

[0013] In some embodiments, the fitting unit fits the weld point cloud data to obtain the weld points of the straight weld, including: segmenting the weld point cloud data using density clustering with normal vector constraints to obtain a weld point cloud set; the weld point cloud set includes point cloud clusters; performing straight line fitting on each point cloud cluster in the weld point cloud set to obtain a fitted straight line for each point cloud cluster; projecting all points in each point cloud cluster onto the corresponding fitted straight line, and evenly sampling coordinate points at preset intervals within the projection range to generate weld points of the straight weld.

[0014] In some embodiments, the fitting unit uses density clustering with normal vector constraints to segment the weld point cloud data to obtain a weld point cloud set, including: determining whether the point cloud cluster in the weld point cloud data satisfies a preset first condition and a preset second condition; if the point cloud cluster satisfies the preset first condition and the preset second condition, the point cloud cluster is added to the weld point cloud set; if the point cloud cluster does not satisfy the preset first condition or the preset second condition, the point cloud cluster is not added to the weld point cloud set; wherein the preset first condition is that the number of point cloud data within a preset first range of the point cloud cluster is greater than a point cloud data number threshold; the preset second condition is that the angle between the normal vectors of all point cloud data within the preset first range of the point cloud cluster is less than or equal to an angle threshold.

[0015] In some embodiments, the trajectory planning unit performs weld point fine-tuning and posture estimation on the weld points of the weld to obtain the welding trajectory of the straight weld, including: for each weld point in the weld, recording the set of point cloud data of all weld points within a preset second range of the current weld point as a local point cloud set; judging whether a number of planes greater than or equal to a plane number threshold can be segmented from the local point cloud set; if a number of planes greater than or equal to a plane number threshold cannot be segmented from the local point cloud set, then no weld point fine-tuning is performed on the current weld point; if a number of planes greater than or equal to a plane number threshold can be segmented from the local point cloud set, then calculating the intersection of the segmented planes; projecting the current weld point onto the intersection to obtain a projection point; using the projection point as the weld point after weld point fine-tuning of the current weld point; performing posture estimation on all weld points that have completed weld point fine-tuning to obtain the welding trajectory of the straight weld.

[0016] In some embodiments, the trajectory planning unit estimates the posture of all weld points fine-tuned in the complete weld points to obtain the welding trajectory of the straight weld, and further includes: for each weld point fine-tuned in the complete weld points, normalizing the direction vector of the intersection line of the current weld point to obtain the x-axis basis vector; in the robot arm base coordinate system, taking the point on one side of the weld of the workpiece as the viewpoint, calculating the centroid of the point set of the neighborhood point cloud of the current weld point projected to the intersection line, and obtaining the direction vector of the viewpoint pointing to the centroid, which is recorded as the first direction vector; calculating the normal vector set of the plane, and judging whether there is a normal vector in the normal vector set that is an obtuse angle with the first direction vector; if there is a normal vector with an obtuse angle with the first direction vector, If the angle between the first direction vector and the normal vector is an obtuse angle, the direction of the normal vector whose angle with the first direction vector is an obtuse angle is reversed; if there is no normal vector whose angle with the first direction vector is an obtuse angle, all normal vectors in the normal vector set are summed to obtain a second direction vector; the component of the second direction vector perpendicular to the x-axis basis vector is calculated, and the z-axis basis vector is obtained after normalization; the y-axis basis vector is determined according to the x-axis basis vector and the z-axis basis vector; the coordinates of each weld point in the weld seam are used as the origin of the coordinate system, and the x-axis basis vector, the z-axis basis vector, and the y-axis basis vector constitute the coordinate system orientation to form the welding trajectory of the straight weld.

[0017] Matching the above-mentioned device, the present invention further provides a welding robot, comprising: the welding trajectory planning device of the welding robot described above.

[0018] In accordance with the above method, the present invention provides a storage medium on another aspect, wherein the storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the welding trajectory planning method of the welding robot described above.

[0019] In accordance with the above method, the present invention provides a computer program product on another aspect, wherein the computer program product includes a computer program, and when the computer program product is processed and executed, the steps of the welding trajectory planning method of the above welding robot are implemented.

[0020] The solution of the present invention inputs the workpiece's point cloud data into a point cloud semantic segmentation model to obtain weld point cloud data for the workpiece's linear welds. This weld point cloud data is then fitted to obtain the weld points for the linear welds. Finally, the weld points are fine-tuned and their poses estimated to obtain the weld trajectory for the linear welds. This not only enables precise identification and extraction of the point cloud for linear welds, improving the efficiency and accuracy of weld detection, but also enhances the precision and adaptability of welding trajectory planning.

[0021] Other features and advantages of the present invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the present invention.

[0022] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic flow chart of an embodiment of a welding trajectory planning method for a welding robot according to the present invention;

[0024] Figure 2 Schematic diagram of the structure of an embodiment of a welding trajectory planning device for a welding robot of the present invention;

[0025] Figure 3 This is the SPSNet architecture diagram of the weld point cloud semantic segmentation model;

[0026] Figure 4 Schematic diagram of the topological feature learning module in the weld point cloud semantic segmentation model;

[0027] Figure 5 Schematic diagram of the geometric feature learning module in the weld point cloud semantic segmentation model;

[0028] Figure 6 Flowchart for weld fitting;

[0029] Figure 7 Flowchart for solder joint fine-tuning and pose estimation.

[0030] In conjunction with the accompanying drawings, the reference numerals in the embodiments of the present invention are as follows:

[0031] 101 - acquisition unit; 102 - semantic segmentation unit; 103 - fitting unit; 104 - trajectory planning unit. DETAILED DESCRIPTION

[0032] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0033] In modern manufacturing, welding technology is a vital process for ensuring product quality and safety, and is particularly widely used in the automotive, aerospace, and shipbuilding industries. The premise for welding trajectory planning is to obtain weld position information through weld detection. Traditional linear weld detection methods rely primarily on manual inspection or contact sensor technology. However, manual inspection is inefficient and susceptible to subjective factors, making it difficult to guarantee the accuracy and consistency of detection results. While contact sensor-based detection methods can provide a certain degree of automation support, they suffer from issues such as insufficient detection accuracy and poor adaptability. Robotic welding, on the other hand, struggles to adapt to workpiece geometric deviations or weld position changes, resulting in unstable welding quality.

[0034] Therefore, the present invention provides a welding trajectory planning method for a welding robot. This method acquires high-precision point cloud data of the workpiece to be welded and then constructs a point cloud semantic segmentation model (SPSNet) based on deep learning technology. This model learns the geometric and topological features of weld seams, enabling accurate identification and efficient extraction of linear weld point clouds from various weld types in complex environments. Furthermore, based on the extracted linear weld point clouds, the method dynamically adjusts and optimizes the welding trajectory through weld seam fitting and local weld point fine-tuning, effectively adapting to workpiece geometric deviations and weld position variations, thereby improving welding trajectory accuracy.

[0035] According to an embodiment of the present invention, a welding trajectory planning method for a welding robot is provided. Figure 1 FIG. 1 is a flow chart of an embodiment of the method of the present invention. The welding trajectory planning method of the welding robot may include steps S110 to S140.

[0036] In step S110 , point cloud data of the workpiece is acquired.

[0037] The 3D camera can be used to collect three-dimensional coordinate point sets on the workpiece surface to form point cloud data containing weld and non-weld areas, which serves as the basic input for subsequent processing.

[0038] In step S120 , the point cloud data of the workpiece is input into a preset point cloud semantic segmentation model to obtain weld point cloud data of the linear weld of the workpiece.

[0039] The point cloud semantic segmentation model is the SPSNet model. The SPSNet model is used to fuse the topological and geometric features of the weld, and deep learning is used to distinguish weld and non-weld point clouds. The weld point cloud data of straight welds is accurately identified, which solves the problems of large subjective errors in manual detection and poor adaptability of contact sensors, and realizes the automatic and high-precision extraction of weld positions.

[0040] like Figure 3 As shown, the point cloud semantic segmentation model includes: a topological feature learning module, a geometric feature learning module, a feature fusion module, a label prediction module, and a point cloud extraction module.

[0041] The topological feature learning module is used to extract the topological features of the local connection relationship of the point cloud and characterize the local structural information of the weld. Its principle is to use a two-layer EdgeConv (graph neural network convolution) to build a point cloud neighborhood graph (knn graph), calculate the edge features between points, and map the edge features to high-dimensional features through MLP. After maximum pooling, the global topological features f are output. top , with dimensions of 1×512.

[0042] The geometric feature learning module is used to extract the global morphological features of the point cloud and characterize the overall direction of the weld. Its principle is to use a 4-layer MLP (64, 64, 128, 512) to map the original point cloud coordinates into high-dimensional features, compress the feature dimensions through maximum pooling, and output the global geometric features f geo , with dimensions of 1×512.

[0043] The feature fusion module is used to stitch topological and geometric features to form a complete representation containing local and global information. top and f geo Get the 1×1024 splicing feature, map and copy the feature through MLP, and output the fusion feature f fus , the dimension is n×1024 (n is the number of point cloud points).

[0044] The label prediction module converts the fused features into point cloud class probabilities, distinguishing welds from non-weld points. This involves dimensionality reduction through four fully connected layers (512, 256, 128, 64), ultimately mapping them to two-dimensional features (c = 2 categories) through the fully connected layers. A softmax function is then applied to determine the probability that each point belongs to a weld.

[0045] The point cloud extraction module selects weld point clouds based on probability, outputting a clean set of straight weld coordinates. Specifically, it sets a probability threshold (e.g., 0.5), retains points with probabilities above the threshold, and extracts their 3D coordinates.

[0046] The pre-training process for the point cloud semantic segmentation model involves using a 3D camera to collect point cloud data of a certain number of different types of welded workpieces. The point clouds are manually annotated with semantic labels, and the workpiece point clouds are classified into two categories: weld point clouds and other point clouds. Point cloud data augmentation methods such as random rotation, scaling, translation, and dithering are then used to increase the number and diversity of training samples, creating model training and test datasets. The weld point clouds include point clouds from various welding methods, such as linear, curved, and circular welds. Point cloud data augmentation increases the number and diversity of training samples, thereby improving the model's generalization and segmentation accuracy.

[0047] In some embodiments, in step S120, the point cloud data of the workpiece is input into a preset point cloud semantic segmentation model to obtain the weld point cloud data of the linear weld of the workpiece, which includes steps S210 to S250.

[0048] Step S210 : inputting the point cloud data of the workpiece into the topological feature learning module to obtain global topological features.

[0049] like Figure 3 As shown in Figure 1, the point cloud data of the workpiece will be input into the topological feature learning module and the geometric feature learning module respectively. The topological feature reflects the connection relationship between points and can effectively characterize the local structural information of the weld. The logic of the topological feature learning module is as follows: Figure 4 As shown in the figure, the topological feature learning module consists of two EdgeConvs, which are convolution operations in graph neural networks. The first EdgeConv extracts weld topological features of size (n×64) from the input point cloud of size (n×3). For the input point cloud, a k-nearest neighbor graph structure (knn graph) is first constructed to obtain point cloud edge features of size (n×k×6). Then, a multi-layer perceptron (MLP) composed of two fully connected layers is used to map the point cloud edge features into high-dimensional features of size (n×k×64). After maximum pooling, the point cloud topological features f of size (n×64) are obtained. edge1 .

[0050] Assume x i ,i∈n is a point in the input point cloud, x i ,i∈k is x i One of the k nearest neighbors of x i and x j The edge feature hθ(x i ,x j ) is represented as follows:

[0051] hθ(x i ,x j )=hθ(x i ,xi -x j );

[0052] Where θ is a learnable parameter.

[0053] The second EdgeConv takes the input f edge1 Further mapped to the (n×128) point cloud topology feature f edge2 , compared to f edge1 , f edge2 It has a larger receptive field. In order to obtain multi-scale weld structure information, splicing f edge1 and f edge2 , and use the multi-layer perceptron (mlp) to map it into a high-dimensional feature of size (n×512), and finally obtain the weld topology feature f of size (1×512) through maximum pooling top .

[0054] Step S220: inputting the point cloud data of the workpiece into the geometric feature learning module to obtain global geometric features.

[0055] Weld topological features f top It can characterize the local structural information of the weld, but lacks the global structural information of the weld. Therefore, a weld geometric feature learning module is constructed to extract the global geometric features of the weld. The logic of the geometric feature learning module is as follows: Figure 5 As shown in the figure, for an input point cloud of size (n×3), four multi-layer perceptrons (MLPs) are used to map the point cloud into a high-dimensional feature of (n×512), and then the maximum pooling is used to finally obtain the weld geometry feature f of size (1×512). geo , characterizing the global geometric structure information of the weld.

[0056] Step S230: input the global topological features and the global geometric features into the feature fusion module to obtain fused features.

[0057] In the feature fusion module, the extracted weld topology features f are further spliced ​​together. top and weld geometry f geo , we get a concatenated feature of size (1×1024), and then use a multi-layer perceptron to map the feature into a feature of size (1×1024), copy the feature n times, and get a fusion feature f of size (n×1024). fus .f fus It contains both local and global structural information of the weld, and can identify the weld more accurately.

[0058] Step S240 : Processing the fusion features using the label prediction module to obtain a probability that each point cloud data in the point cloud data of the workpiece belongs to the point cloud data of a linear weld.

[0059] The fusion feature f fus Input the label prediction module to learn the label information of the point cloud. In the label prediction module, a multi-layer perceptron composed of 4 fully connected layers is used to sequentially transform the fusion feature f fus The feature dimensions are mapped to 512, 256, 128, and 64 dimensions, resulting in features of size (n × 64). Finally, a fully connected layer is used to map the feature dimensions to c, where c represents the category of the point cloud and can be set to 2, resulting in features of size (n × 2). A softmax function is applied to this feature to convert each output value into a probability between 0 and 1, obtaining the label category information of the point cloud.

[0060] In step S250 , the point cloud extraction module extracts the complete weld point cloud data of the linear weld according to the probability that each point cloud data belongs to the point cloud data of the linear weld.

[0061] In the point cloud extraction module, weld point clouds are extracted based on point cloud category probabilities. By integrating topological and geometric features, linear welds in complex workpieces can be accurately identified. Label prediction and point cloud extraction replace manual labor, shortening weld inspection time and improving efficiency.

[0062] In step S130, the weld point cloud data is fitted to obtain weld points of the linear weld.

[0063] The fitting process can filter out noise points, adapt to the uneven surface of the workpiece or point cloud noise, avoid the mismatch of the preset trajectory caused by the geometric deviation of the workpiece, and make the weld point fit the actual weld direction.

[0064] In some embodiments, in step S130, the specific process of fitting the weld point cloud data to obtain the weld points of the linear weld includes: steps S310 to S330.

[0065] Step S310 , segmenting the weld point cloud data using density clustering with normal vector constraints to obtain a weld point cloud set; the weld point cloud set includes point cloud clusters.

[0066] In some embodiments, in step S310, the specific process of segmenting the weld point cloud data using density clustering with normal vector constraints to obtain a weld point cloud set includes: determining whether the point cloud cluster in the weld point cloud data satisfies a preset first condition and a preset second condition; if the point cloud cluster satisfies the preset first condition and the preset second condition, adding the point cloud cluster to the weld point cloud set; if the point cloud cluster does not satisfy the preset first condition or the preset second condition, not adding the point cloud cluster to the weld point cloud set; wherein the preset first condition is that the number of point cloud data within a preset first range of the point cloud cluster is greater than a point cloud data number threshold; the preset second condition is that the angle between the normal vectors of all point cloud data within the preset first range of the point cloud cluster is less than or equal to an angle threshold.

[0067] The point cloud of a linear weld should exhibit high-density clustering in localized areas, while noise points or non-weld points are typically sparsely distributed. Density filtering can be used to filter out isolated noise points and retain the true weld area. For example, random noise points on the workpiece surface typically have fewer than 10 points within a 0.01m neighborhood and are therefore eliminated. The surface normal vectors of a linear weld should generally point in the same direction, while the normal vector directions in non-weld areas of complex workpieces vary significantly. Normal vector angle constraints can be used to ensure that the point cloud cluster conforms to the geometric characteristics of the linear weld and exclude non-linear structures.

[0068] Specifically, the weld point cloud is segmented using point cloud density clustering with normal vector constraints to obtain independent straight weld point clouds. Among them, point cloud density clustering with normal vector constraints is a clustering method that considers both point cloud density information and normal vector direction information. The density information represents the local density of the point cloud as the number of points within the radius neighborhood. The more points there are, the higher the density. The density of the point cloud in the same point cloud cluster, that is, the number of points within the radius neighborhood, should be greater than the point count threshold, such as the number of points within a radius of 0.01m is greater than 10. The normal vector directions of the points in the same point cloud cluster are roughly in the same direction. If the normal vector angle between points does not exceed 20 degrees, it is considered that the normal vector directions are roughly in the same direction. When the number of points in the point cloud cluster obtained after segmentation is greater than 50, the point cloud cluster is considered to be a valid weld point cloud and is retained to obtain an independent straight weld point cloud S={s1,…,s t}, where t represents the number of straight weld point clouds.

[0069] In complex welded workpieces, different weld segment point clouds are interconnected and have similar density. Point cloud density alone cannot segment these weld segment point clouds into independent ones. Since the normal vectors of different weld segment point clouds vary significantly, adding normal vector constraints and considering both point cloud density and normal vector information allows segmentation of interconnected weld segment point clouds into independent ones. Points with density below a threshold are treated as noise and removed, thus filtering out noise.

[0070] By identifying valid weld point cloud clusters based on density and normal vector conditions, the extraction accuracy of linear welds in complex workpieces is improved and the reliability of segmentation results is ensured.

[0071] Step S320 , performing straight line fitting on each point cloud cluster in the weld point cloud set to obtain a fitting straight line for each point cloud cluster.

[0072] Linear fitting is to fit a straight line to each point cloud cluster using the random sampling consensus algorithm (RANSAC). Specifically, for the weld point cloud s in the weld point cloud set S i ,i∈(1,t)n, and a straight line is fitted to the point cloud using the random sampling consistency algorithm to obtain the straight line equation. The internal point screening mechanism of the random sampling consistency algorithm dynamically adapts to the local bending or deformation of the weld, making the fitted straight line closer to the actual weld direction.

[0073] In step S330 , all points in each point cloud cluster are projected onto the corresponding fitting line, and coordinate points are uniformly sampled at preset intervals within the projection range to generate weld points of the linear weld.

[0074] The weld point cloud i Projecting onto the line, sampling coordinate points at equal intervals within the point cloud projection range, and obtaining the weld point set P = {p1,…,p m}, where m represents the number of weld points. By converting continuous straight lines into discrete weld points that can be executed by the robot, the trajectory accuracy and execution efficiency are balanced.

[0075] Figure 6 The flow chart for weld fitting is as follows: Figure 6 As shown, the method includes:

[0076] Step 1: Use point cloud density clustering with normal vector constraints to segment the weld point cloud and obtain independent linear weld point clouds.

[0077] Step 2: Determine whether the point cloud meets the conditions of density exceeding the threshold and normal vector direction being consistent. If so, proceed to step 3; if not, discard the point cloud.

[0078] Step 3: For the obtained straight weld point cloud S={s1,…,s t}, use the random sampling consistency algorithm to perform straight line fitting on the point cloud and get the straight line equation. i Projecting onto the line, sampling coordinate points at equal intervals within the point cloud projection range, and obtaining the weld point set P = {p1,…,p m}.

[0079] In step S140 , the weld points are fine-tuned and the position and posture are estimated to obtain the welding trajectory of the straight weld.

[0080] Fine-tuning weld points can address global fitting errors, adapt to local workpiece deformation, and avoid unstable welding quality caused by workpiece geometric deviations, ensuring that weld points precisely fall on the actual weld seam. Pose estimation is used to assign weld point orientation information, aligning the welding gun's posture with the weld surface normal. This avoids welding defects caused by a fixed welding gun angle within a preset trajectory and improves weld quality consistency.

[0081] By replacing manual annotation and contact sensors with 3D point clouds and SPSNet models, we address accuracy and adaptability issues in the inspection process and provide a reliable data foundation for trajectory planning. Intelligent conversion from point clouds to trajectories and pose estimation improve the accuracy of trajectory planning and enhance the welding adaptability of complex workpieces.

[0082] In some embodiments, in step S140, the specific process of performing weld point fine-tuning and pose estimation on the weld point to obtain the welding trajectory of the straight weld includes: steps S410 to S450.

[0083] Step S410 : for each weld point in the weld seam, a set of point cloud data of all weld points within a preset second range of the current weld point is recorded as a local point cloud set.

[0084] The global point cloud may contain distant noise points or non-weld structures, which can easily lead to fine-tuning errors if directly processed. Therefore, fine-tuning of weld points is necessary. First, for each weld point, all point cloud data within a preset second range (e.g., a radius of 0.02m) is searched to form a local point cloud collection. This narrowed processing range improves computational efficiency, reduces unnecessary point cloud interference, and accurately captures the true weld shape near the weld point.

[0085] Step S420 , determining whether planes whose number is greater than or equal to a plane number threshold can be segmented from the local point cloud set.

[0086] Step S430: If the number of planes greater than or equal to the plane number threshold cannot be segmented from the local point cloud set, no weld fine-tuning is performed on the current weld.

[0087] A linear weld can be approximated locally as the intersection of two planes. If sufficient planes cannot be segmented, the local point cloud may contain noise or non-weld structures. A random sampling consensus algorithm is used to segment planes from the local point cloud set, determining whether a number of planes ≥ a threshold (usually two) can be segmented.

[0088] Step S440: If a number of planes greater than or equal to the plane number threshold can be segmented from the local point cloud set, the intersection of the segmented planes is calculated; the current weld point is projected onto the intersection line to obtain a projection point; and the projection point is used as the weld point after the weld point is fine-tuned.

[0089] If two or more planes are segmented, the intersection of the two planes is calculated and the weld point is projected onto the intersection to obtain the projection point, which serves as the fine-tuned weld point position. The intersection of the two planes is now the true position of the local weld. The projection operation corrects this deviation, ensuring that the weld point falls precisely on the actual weld.

[0090] Specifically, the weld point cloud s i There may be noise points in the image. The straight line fitted by the random sampling consistency algorithm does not completely coincide with the weld, resulting in some weld points not being on the weld. The weld points need to be fine-tuned. For each weld point in P, first use the radius neighbor search to search the local point cloud within a radius of 0.02m. Then use the random sampling consistency algorithm to segment the two planes that make up the weld from the local neighborhood point cloud. If the number of segmented planes is less than 2 (such as the workpiece point cloud is incomplete), the weld point will not be fine-tuned. Otherwise, calculate the intersection line of the two planes and the plane normal vector. In the local range, the intersection line of the two planes can be regarded as a straight line that coincides with the weld. Project the weld point onto the intersection line to obtain the projection point set R={r1,…,r k}, calculate the plane normal vector, and get the plane normal vector set V={v1,…,v k}, where k represents the number of solder joints after projection. Finally, the projection point set R={r1,…,r k} to perform straight line fitting, and set P={p1,…,p m All solder points in} are projected onto the straight line to obtain the fine-tuned solder point P r ={p r 1,…,p r m By fine-tuning the local weld and projecting the weld points in P onto the weld, it is possible to overcome the geometric deviation of the workpiece and the change in the weld position, and significantly improve the position accuracy of the weld points.

[0091] Step S450 , performing pose estimation on all weld points that have completed weld point fine-tuning to obtain the welding trajectory of the straight weld.

[0092] Pose estimation involves constructing a workpiece coordinate system at the weld point. By performing local point cloud analysis, plane segmentation, and fine-tuning intersection projection on the weld point, combined with pose estimation to construct a workpiece coordinate system, the weld point position can be accurately corrected and dynamically adapted to the weld geometry. This effectively overcomes the effects of workpiece geometric deviations and weld position variations, significantly improving the accuracy and adaptability of welding trajectory planning and providing reliable technical support for intelligent robotic welding.

[0093] In some embodiments, in step S450, the posture estimation of all welds that have completed weld fine-tuning is performed to obtain the specific process of the welding trajectory of the straight weld, including: for each weld that has completed weld fine-tuning, the direction vector of the intersection line of the current weld is normalized to obtain the x-axis basis vector; in the robot arm base coordinate system, the point located on one side of the weld of the workpiece is used as the viewpoint, and the center of mass of the point set of the neighborhood point cloud of the current weld point projected to the intersection line is calculated to obtain the direction vector of the viewpoint pointing to the center of mass, which is recorded as the first direction vector; the normal vector set of the plane is calculated to determine whether there is a normal vector in the normal vector set that has an obtuse angle with the first direction vector; if there is a normal vector, the first direction vector is determined to be ... When the normal vector forms an obtuse angle with the first direction vector, the direction of the normal vector forming an obtuse angle with the first direction vector is reversed; if there is no normal vector forming an obtuse angle with the first direction vector, all normal vectors in the normal vector set are summed to obtain a second direction vector; the component of the second direction vector perpendicular to the x-axis basis vector is calculated, and the z-axis basis vector is obtained after normalization; the y-axis basis vector is determined based on the x-axis basis vector and the z-axis basis vector; the coordinates of each weld point in the weld are used as the origin of the coordinate system, and the x-axis basis vector, the z-axis basis vector, and the y-axis basis vector constitute the orientation of the coordinate system to form the welding trajectory of the straight weld.

[0094] Specifically, the direction vector of the straight line fitted by the projection point set R is first used as the direction vector of the x-axis of the weld workpiece coordinate system, and the x-axis basis vector v is obtained after normalization. x Then, in the robot base coordinate system, select a point in the space inside the workpiece (on the weld side) as the viewpoint p v . Calculate the projection point set R={r1,…,r k}'s center of mass point p center , obtained from the viewpoint p v Point to p center The direction vector v vc . Then calculate the vector v vc The angle between the normal vector and each normal vector in the normal vector set V is an acute angle, which means that the normal vector direction is at the same angle as v. vc The direction is consistent, otherwise the direction of the normal vector is reversed to obtain a set of plane normal vectors V with a uniform direction. r ={v r 1,…,v r m}. Then V r Add the normal vectors in to get the direction vector v normal , calculate its basis vector v perpendicular to the x-axis x The components of the workpiece coordinate system z-axis basis vector v is obtained after normalization. zFinally, based on the right-hand rule, the y-axis basis vector v of the workpiece coordinate system is calculated y On the same straight weld, the workpiece coordinate system of different weld points has the same coordinate axis orientation, and the coordinate origin is the weld point's own coordinate, thus obtaining the precise welding trajectory of the weld in the robot base coordinate system.

[0095] By normalizing the intersection direction vector of the fine-tuned weld point to determine the x-axis, combining the viewpoint and centroid vector to unify the normal vector direction and generate the z-axis, and then determining the y-axis based on the right-hand system, a workpiece coordinate system with the weld point as the origin is constructed. The position and posture of each weld point can be accurately defined, and the welding trajectory can be adaptive to the geometric deviation of the workpiece, ensuring that the welding gun welds along the direction of the weld and perpendicular to the weld surface, significantly improving the accuracy of the welding trajectory and process adaptability.

[0096] Figure 7 The flowchart of solder joint fine-tuning and pose estimation is as follows: Figure 7 As shown, the method includes:

[0097] Step 11: For each weld point in the weld, use radius neighborhood search to search the local point cloud within a radius of 0.02m, and then use the random sampling consistency algorithm to segment the plane that constitutes the weld from the local neighborhood point cloud.

[0098] Step 12: Determine whether the number of planes segmented is less than 2. If so, skip the solder joint; otherwise, proceed to step 13.

[0099] Step 13: Calculate the intersection line of the two planes and the plane normal vectors, and use the sum of the plane normal vectors as the z-axis vector of the welding point.

[0100] Step 14: Project the solder point onto the plane intersection line to obtain the projection point, which is the solder point after fine-tuning.

[0101] Step 15: Project the weld point onto the straight line fitted by the projection point, and use the direction vector of the straight line as the x-axis vector of the weld.

[0102] Step 16: Calculate the z-axis vector of the weld and obtain the position of each weld point based on the x-axis vector to obtain the weld trajectory.

[0103] Through weld fitting and local fine-tuning technology of weld points, dynamic adjustment and optimization of welding trajectory can be achieved, the accuracy of welding trajectory can be improved, and the problem that the preset welding trajectory of the robot is difficult to adapt to the geometric deviation of the workpiece and the change of weld position can be solved, providing technical support for the widespread application of robot welding technology.

[0104] Using the technical solution of this embodiment, the workpiece's point cloud data is input into a point cloud semantic segmentation model to obtain weld point cloud data for the workpiece's linear welds. This weld point cloud data is then fitted to obtain the weld points for the linear welds. Finally, the weld points are fine-tuned and their poses estimated to obtain the weld trajectory for the linear welds. This not only enables accurate identification and extraction of linear weld point clouds, improving the efficiency and accuracy of weld inspection, but also enhances the precision and adaptability of welding trajectory planning.

[0105] According to an embodiment of the present invention, a welding trajectory planning device for a welding robot corresponding to the welding trajectory planning method for the welding robot is also provided. Figure 2 The schematic structural diagram of an embodiment of the device of the present invention is shown in FIG. The welding trajectory planning device of the welding robot may include: an acquisition unit 101 , a semantic segmentation unit 102 , a fitting unit 103 , and a trajectory planning unit 104 .

[0106] The acquiring unit 101 is configured to acquire point cloud data of a workpiece.

[0107] The 3D camera can be used to collect three-dimensional coordinate point sets on the workpiece surface to form point cloud data containing weld and non-weld areas, which serves as the basic input for subsequent processing.

[0108] The semantic segmentation unit 102 is configured to input the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain weld point cloud data of the linear weld of the workpiece.

[0109] The point cloud semantic segmentation model is the SPSNet model. The SPSNet model is used to fuse the topological and geometric features of the weld, and deep learning is used to distinguish weld and non-weld point clouds. The weld point cloud data of straight welds is accurately identified, which solves the problems of large subjective errors in manual detection and poor adaptability of contact sensors, and realizes the automatic and high-precision extraction of weld positions.

[0110] like Figure 3 As shown, the point cloud semantic segmentation model includes: a topological feature learning module, a geometric feature learning module, a feature fusion module, a label prediction module, and a point cloud extraction module.

[0111] The topological feature learning module is used to extract the topological features of the local connection relationship of the point cloud and characterize the local structural information of the weld. Its principle is to use a two-layer EdgeConv (graph neural network convolution) to build a point cloud neighborhood graph (knn graph), calculate the edge features between points, and map the edge features to high-dimensional features through MLP. After maximum pooling, the global topological features f are output. top , with dimensions of 1×512.

[0112] The geometric feature learning module is used to extract the global morphological features of the point cloud and characterize the overall direction of the weld. Its principle is to use a 4-layer MLP (64, 64, 128, 512) to map the original point cloud coordinates into high-dimensional features, compress the feature dimensions through maximum pooling, and output the global geometric features f geo , with dimensions of 1×512.

[0113] The feature fusion module is used to stitch topological and geometric features to form a complete representation containing local and global information. top and f geo Get the 1×1024 splicing feature, map and copy the feature through MLP, and output the fusion feature f fus , the dimension is n×1024 (n is the number of point cloud points).

[0114] The label prediction module converts the fused features into point cloud class probabilities, distinguishing welds from non-weld points. This involves dimensionality reduction through four fully connected layers (512, 256, 128, 64), ultimately mapping them to two-dimensional features (c = 2 categories) through the fully connected layers. A softmax function is then applied to determine the probability that each point belongs to a weld.

[0115] The point cloud extraction module selects weld point clouds based on probability, outputting a clean set of straight weld coordinates. Specifically, it sets a probability threshold (e.g., 0.5), retains points with probabilities above the threshold, and extracts their 3D coordinates.

[0116] The pre-training process for the point cloud semantic segmentation model involves using a 3D camera to collect point cloud data of a certain number of different types of welded workpieces. The point clouds are manually annotated with semantic labels, and the workpiece point clouds are classified into two categories: weld point clouds and other point clouds. Point cloud data augmentation methods such as random rotation, scaling, translation, and dithering are then used to increase the number and diversity of training samples, creating model training and test datasets. The weld point clouds include point clouds from various welding methods, such as linear, curved, and circular welds. Point cloud data augmentation increases the number and diversity of training samples, thereby improving the model's generalization and segmentation accuracy.

[0117] In some embodiments, the semantic segmentation unit 102 inputs the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain weld point cloud data of a linear weld of the workpiece, including:

[0118] The semantic segmentation unit 102 is further configured to input the point cloud data of the workpiece into the topological feature learning module to obtain global topological features.

[0119] like Figure 3As shown in Figure 1, the point cloud data of the workpiece will be input into the topological feature learning module and the geometric feature learning module respectively. The topological feature reflects the connection relationship between points and can effectively characterize the local structural information of the weld. The logic of the topological feature learning module is as follows: Figure 4 As shown in the figure, the topological feature learning module consists of two EdgeConvs, which are convolution operations in graph neural networks. The first EdgeConv extracts weld topological features of size (n×64) from the input point cloud of size (n×3). For the input point cloud, a k-nearest neighbor graph structure (knn graph) is first constructed to obtain point cloud edge features of size (n×k×6). Then, a multi-layer perceptron (MLP) composed of two fully connected layers is used to map the point cloud edge features into high-dimensional features of size (n×k×64). After maximum pooling, the point cloud topological features f of size (n×64) are obtained. edge1 .

[0120] Assume x i ,i∈n is a point in the input point cloud, x i ,i∈k is x i One of the k nearest neighbors of x i and x j The edge feature hθ(x i ,x j ) is represented as follows:

[0121] hθ(x i ,x j )=hθ(x i ,x i -x j );

[0122] Where θ is a learnable parameter.

[0123] The second EdgeConv takes the input f edge1 Further mapped to the (n×128) point cloud topology feature f edge2 , compared to f edge1 , f edge2 It has a larger receptive field. In order to obtain multi-scale weld structure information, splicing f edge1 and f edge2 , and use the multi-layer perceptron (mlp) to map it into a high-dimensional feature of size (n×512), and finally obtain the weld topology feature f of size (1×512) through maximum pooling top .

[0124] The semantic segmentation unit 102 is further configured to input the point cloud data of the workpiece into the geometric feature learning module to obtain global geometric features.

[0125] Weld topological features ftop It can characterize the local structural information of the weld, but lacks the global structural information of the weld. Therefore, a weld geometric feature learning module is constructed to extract the global geometric features of the weld. The logic of the geometric feature learning module is as follows: Figure 5 As shown in the figure, for an input point cloud of size (n×3), four multi-layer perceptrons (MLPs) are used to map the point cloud into a high-dimensional feature of (n×512), and then the maximum pooling is used to finally obtain the weld geometry feature f of size (1×512). geo , characterizing the global geometric structure information of the weld.

[0126] The semantic segmentation unit 102 is further configured to input the global topological features and the global geometric features into the feature fusion module to obtain fused features.

[0127] In the feature fusion module, the extracted weld topology features f are further spliced ​​together. top and weld geometry f geo , we get a concatenated feature of size (1×1024), and then use a multi-layer perceptron to map the feature into a feature of size (1×1024), copy the feature n times, and get a fusion feature f of size (n×1024). fus .f fus It contains both local and global structural information of the weld, and can identify the weld more accurately.

[0128] The semantic segmentation unit 102 is further configured to process the fusion features using the label prediction module to obtain a probability that each point cloud data in the point cloud data of the workpiece belongs to the point cloud data of a linear weld.

[0129] The fusion feature f fus Input the label prediction module to learn the label information of the point cloud. In the label prediction module, a multi-layer perceptron composed of 4 fully connected layers is used to sequentially transform the fusion feature f fus The feature dimensions are mapped to 512, 256, 128, and 64 dimensions, resulting in features of size (n × 64). Finally, a fully connected layer is used to map the feature dimensions to c, where c represents the category of the point cloud and can be set to 2, resulting in features of size (n × 2). A softmax function is applied to this feature to convert each output value into a probability between 0 and 1, obtaining the label category information of the point cloud.

[0130] The semantic segmentation unit 102 is further configured as the point cloud extraction module to extract the complete weld point cloud data of the straight weld according to the probability that each point cloud data belongs to the point cloud data of the straight weld.

[0131] In the point cloud extraction module, weld point clouds are extracted based on point cloud category probabilities. By integrating topological and geometric features, linear welds in complex workpieces can be accurately identified. Label prediction and point cloud extraction replace manual labor, shortening weld inspection time and improving efficiency.

[0132] The fitting unit 103 is configured to fit the weld point cloud data to obtain weld points of the linear weld.

[0133] The fitting process can filter out noise points, adapt to the uneven surface of the workpiece or point cloud noise, avoid the mismatch of the preset trajectory caused by the geometric deviation of the workpiece, and make the weld point fit the actual weld direction.

[0134] In some embodiments, the fitting unit 103 fits the weld point cloud data to obtain the weld points of the linear weld, including:

[0135] The fitting unit 103 is further configured to segment the weld point cloud data using density clustering with normal vector constraints to obtain a weld point cloud set; the weld point cloud set includes point cloud clusters.

[0136] In some embodiments, the fitting unit 103 uses density clustering with normal vector constraints to segment the weld point cloud data to obtain a weld point cloud set, including: determining whether the point cloud cluster in the weld point cloud data satisfies a preset first condition and a preset second condition; if the point cloud cluster satisfies the preset first condition and the preset second condition, the point cloud cluster is added to the weld point cloud set; if the point cloud cluster does not satisfy the preset first condition or the preset second condition, the point cloud cluster is not added to the weld point cloud set; wherein the preset first condition is that the number of point cloud data within a preset first range of the point cloud cluster is greater than a point cloud data number threshold; the preset second condition is that the angle between the normal vectors of all point cloud data within the preset first range of the point cloud cluster is less than or equal to an angle threshold.

[0137] The point cloud of a linear weld should exhibit high-density clustering in localized areas, while noise points or non-weld points are typically sparsely distributed. Density filtering can be used to filter out isolated noise points and retain the true weld area. For example, random noise points on the workpiece surface typically have fewer than 10 points within a 0.01m neighborhood and are therefore eliminated. The surface normal vectors of a linear weld should generally point in the same direction, while the normal vector directions in non-weld areas of complex workpieces vary significantly. Normal vector angle constraints can be used to ensure that the point cloud cluster conforms to the geometric characteristics of the linear weld and exclude non-linear structures.

[0138] Specifically, the weld point cloud is segmented using point cloud density clustering with normal vector constraints to obtain independent straight weld point clouds. Among them, point cloud density clustering with normal vector constraints is a clustering method that considers both point cloud density information and normal vector direction information. The density information represents the local density of the point cloud as the number of points within the radius neighborhood. The more points there are, the higher the density. The density of the point cloud in the same point cloud cluster, that is, the number of points within the radius neighborhood, should be greater than the point count threshold, such as the number of points within a radius of 0.01m is greater than 10. The normal vector directions of the points in the same point cloud cluster are roughly in the same direction. If the normal vector angle between points does not exceed 20 degrees, it is considered that the normal vector directions are roughly in the same direction. When the number of points in the point cloud cluster obtained after segmentation is greater than 50, the point cloud cluster is considered to be a valid weld point cloud and is retained to obtain an independent straight weld point cloud S={s1,…,s t}, where t represents the number of straight weld point clouds.

[0139] In complex welded workpieces, different weld segment point clouds are interconnected and have similar density. Point cloud density alone cannot segment these weld segment point clouds into independent ones. Since the normal vectors of different weld segment point clouds vary significantly, adding normal vector constraints and considering both point cloud density and normal vector information allows segmentation of interconnected weld segment point clouds into independent ones. Points with density below a threshold are treated as noise and removed, thus filtering out noise.

[0140] By identifying valid weld point cloud clusters based on density and normal vector conditions, the extraction accuracy of linear welds in complex workpieces is improved and the reliability of segmentation results is ensured.

[0141] The fitting unit 103 is further configured to perform straight line fitting on each point cloud cluster in the weld point cloud set to obtain a fitting straight line for each point cloud cluster.

[0142] Linear fitting is to fit a straight line to each point cloud cluster using the random sampling consensus algorithm (RANSAC). Specifically, for the weld point cloud s in the weld point cloud set S i ,i∈(1,t)n, and a straight line is fitted to the point cloud using the random sampling consistency algorithm to obtain the straight line equation. The internal point screening mechanism of the random sampling consistency algorithm dynamically adapts to the local bending or deformation of the weld, making the fitted straight line closer to the actual weld direction.

[0143] The fitting unit 103 is further configured to project all points in each point cloud cluster onto the corresponding fitting line, and uniformly sample coordinate points at preset intervals within the projection range to generate weld points of the linear weld.

[0144] The weld point cloud iProjecting onto the line, sampling coordinate points at equal intervals within the point cloud projection range, and obtaining the weld point set P = {p1,…,p m}, where m represents the number of weld points. By converting continuous straight lines into discrete weld points that can be executed by the robot, the trajectory accuracy and execution efficiency are balanced.

[0145] Figure 6 The flow chart for weld fitting is as follows: Figure 6 As shown, the method includes:

[0146] Step 1: Use point cloud density clustering with normal vector constraints to segment the weld point cloud and obtain independent linear weld point clouds.

[0147] Step 2: Determine whether the point cloud meets the conditions of density exceeding the threshold and normal vector direction being consistent. If so, proceed to step 3; if not, discard the point cloud.

[0148] Step 3: For the obtained straight weld point cloud S={s1,…,s t}, use the random sampling consistency algorithm to perform straight line fitting on the point cloud and get the straight line equation. i Projecting onto the line, sampling coordinate points at equal intervals within the point cloud projection range, and obtaining the weld point set P = {p1,…,p m}.

[0149] The trajectory planning unit 104 is configured to perform weld point fine-tuning and pose estimation on the weld points to obtain a welding trajectory of the straight weld.

[0150] Fine-tuning weld points can address global fitting errors, adapt to local workpiece deformation, and avoid unstable welding quality caused by workpiece geometric deviations, ensuring that weld points precisely fall on the actual weld seam. Pose estimation is used to assign weld point orientation information, aligning the welding gun's posture with the weld surface normal. This avoids welding defects caused by a fixed welding gun angle within a preset trajectory and improves weld quality consistency.

[0151] By replacing manual annotation and contact sensors with 3D point clouds and SPSNet models, we address accuracy and adaptability issues in the inspection process and provide a reliable data foundation for trajectory planning. Intelligent conversion from point clouds to trajectories and pose estimation improve the accuracy of trajectory planning and enhance the welding adaptability of complex workpieces.

[0152] In some embodiments, the trajectory planning unit 104 performs weld point fine-tuning and pose estimation on the weld points to obtain the welding trajectory of the linear weld, including:

[0153] The trajectory planning unit 104 is further configured to record, for each weld point in the weld seam, a set of point cloud data of all weld points within a preset second range of the current weld point as a local point cloud set.

[0154] The global point cloud may contain distant noise points or non-weld structures, which can easily lead to fine-tuning errors if directly processed. Therefore, fine-tuning of weld points is necessary. First, for each weld point, all point cloud data within a preset second range (e.g., a radius of 0.02m) is searched to form a local point cloud collection. This narrowed processing range improves computational efficiency, reduces unnecessary point cloud interference, and accurately captures the true weld shape near the weld point.

[0155] The trajectory planning unit 104 is further configured to determine whether planes whose number is greater than or equal to a plane number threshold can be segmented from the local point cloud set.

[0156] The trajectory planning unit 104 is further configured to not perform weld point fine-tuning on the current weld point if a number of planes greater than or equal to a plane number threshold cannot be segmented from the local point cloud set.

[0157] A linear weld can be approximated locally as the intersection of two planes. If sufficient planes cannot be segmented, the local point cloud may contain noise or non-weld structures. A random sampling consensus algorithm is used to segment planes from the local point cloud set, determining whether a number of planes ≥ a threshold (usually two) can be segmented.

[0158] The trajectory planning unit 104 is further configured to calculate the intersection of the segmented planes if a number of planes greater than or equal to a plane number threshold can be segmented from the local point cloud set; project the current weld point onto the intersection line to obtain a projection point; and use the projection point as the weld point after fine-tuning the weld point.

[0159] If two or more planes are segmented, the intersection of the two planes is calculated and the weld point is projected onto the intersection to obtain the projection point, which serves as the fine-tuned weld point position. The intersection of the two planes is now the true position of the local weld. The projection operation corrects this deviation, ensuring that the weld point falls precisely on the actual weld.

[0160] Specifically, the weld point cloud s iThere may be noise points in the image. The straight line fitted by the random sampling consistency algorithm does not completely coincide with the weld, resulting in some weld points not being on the weld. The weld points need to be fine-tuned. For each weld point in P, first use the radius neighbor search to search the local point cloud within a radius of 0.02m. Then use the random sampling consistency algorithm to segment the two planes that make up the weld from the local neighborhood point cloud. If the number of segmented planes is less than 2 (such as the workpiece point cloud is incomplete), the weld point will not be fine-tuned. Otherwise, calculate the intersection line of the two planes and the plane normal vector. In the local range, the intersection line of the two planes can be regarded as a straight line that coincides with the weld. Project the weld point onto the intersection line to obtain the projection point set R={r1,…,r k}, calculate the plane normal vector, and get the plane normal vector set V={v1,…,v k}, where k represents the number of solder joints after projection. Finally, the projection point set R={r1,…,r k} to perform straight line fitting, and set P={p1,…,p m All solder points in} are projected onto the straight line to obtain the fine-tuned solder point P r ={p r 1,…,p r m By fine-tuning the local weld and projecting the weld points in P onto the weld, it is possible to overcome the geometric deviation of the workpiece and the change in the weld position, and significantly improve the position accuracy of the weld points.

[0161] The trajectory planning unit 104 is further configured to perform pose estimation on all weld points that have completed weld point fine-tuning to obtain the welding trajectory of the straight weld.

[0162] Pose estimation involves constructing a workpiece coordinate system at the weld point. By performing local point cloud analysis, plane segmentation, and fine-tuning intersection projection on the weld point, combined with pose estimation to construct a workpiece coordinate system, the weld point position can be accurately corrected and dynamically adapted to the weld geometry. This effectively overcomes the effects of workpiece geometric deviations and weld position variations, significantly improving the accuracy and adaptability of welding trajectory planning and providing reliable technical support for intelligent robotic welding.

[0163] In some embodiments, the trajectory planning unit 104 performs pose estimation on all welds that have completed weld fine-tuning to obtain the welding trajectory of the straight weld, including: for each weld that has completed weld fine-tuning, normalizing the direction vector of the intersection line of the current weld to obtain the x-axis basis vector; in the robot arm base coordinate system, taking the point on one side of the weld of the workpiece as the viewpoint, calculating the centroid of the point set of the neighborhood point cloud of the current weld projected to the intersection line, and obtaining the direction vector of the viewpoint pointing to the centroid, recorded as the first direction vector; calculating the normal vector set of the plane, and judging whether there is a normal vector in the normal vector set that has an obtuse angle with the first direction vector; if there is a normal vector with an obtuse angle with the first direction vector, If the angle between the first direction vector and the normal vector is an obtuse angle, the direction of the normal vector whose angle with the first direction vector is an obtuse angle is reversed; if there is no normal vector whose angle with the first direction vector is an obtuse angle, all normal vectors in the normal vector set are summed to obtain a second direction vector; the component of the second direction vector perpendicular to the x-axis basis vector is calculated, and the z-axis basis vector is obtained after normalization; the y-axis basis vector is determined based on the x-axis basis vector and the z-axis basis vector; the coordinates of each weld point in the weld seam are used as the origin of the coordinate system, and the x-axis basis vector, the z-axis basis vector, and the y-axis basis vector constitute the coordinate system orientation to form the welding trajectory of the straight weld.

[0164] Specifically, the direction vector of the straight line fitted by the projection point set R is first used as the direction vector of the x-axis of the weld workpiece coordinate system, and the x-axis basis vector v is obtained after normalization. x Then, in the robot base coordinate system, select a point in the space inside the workpiece (on the weld side) as the viewpoint p v . Calculate the projection point set R={r1,…,r k}'s center of mass point p center , obtained from the viewpoint p v Point to p center The direction vector v vc . Then calculate the vector v vc The angle between the normal vector and each normal vector in the normal vector set V is an acute angle, which means that the normal vector direction is at the same angle as v. vc The direction is consistent, otherwise the direction of the normal vector is reversed to obtain a set of plane normal vectors V with a uniform direction. r ={v r 1,…,v r m}. Then V r Add the normal vectors in to get the direction vector v normal , calculate its basis vector v perpendicular to the x-axis x The components of the workpiece coordinate system z-axis basis vector v is obtained after normalization. z Finally, based on the right-hand rule, the y-axis basis vector v of the workpiece coordinate system is calculatedy On the same straight weld, the workpiece coordinate system of different weld points has the same coordinate axis orientation, and the coordinate origin is the weld point's own coordinate, thus obtaining the precise welding trajectory of the weld in the robot base coordinate system.

[0165] By normalizing the intersection direction vector of the fine-tuned weld point to determine the x-axis, combining the viewpoint and centroid vector to unify the normal vector direction and generate the z-axis, and then determining the y-axis based on the right-hand system, a workpiece coordinate system with the weld point as the origin is constructed. The position and posture of each weld point can be accurately defined, and the welding trajectory can be adaptive to the geometric deviation of the workpiece, ensuring that the welding gun welds along the direction of the weld and perpendicular to the weld surface, significantly improving the accuracy of the welding trajectory and process adaptability.

[0166] Figure 7 The flowchart of solder joint fine-tuning and pose estimation is as follows: Figure 7 As shown, the method includes:

[0167] Step 11: For each weld point in the weld, use radius neighborhood search to search the local point cloud within a radius of 0.02m, and then use the random sampling consistency algorithm to segment the plane that constitutes the weld from the local neighborhood point cloud.

[0168] Step 12: Determine whether the number of planes segmented is less than 2. If so, skip the solder joint; otherwise, proceed to step 13.

[0169] Step 13: Calculate the intersection line of the two planes and the plane normal vectors, and use the sum of the plane normal vectors as the z-axis vector of the welding point.

[0170] Step 14: Project the solder point onto the plane intersection line to obtain the projection point, which is the solder point after fine-tuning.

[0171] Step 15: Project the weld point onto the straight line fitted by the projection point, and use the direction vector of the straight line as the x-axis vector of the weld.

[0172] Step 16: Calculate the z-axis vector of the weld and obtain the position of each weld point based on the x-axis vector to obtain the weld trajectory.

[0173] Through weld fitting and local fine-tuning technology of weld points, dynamic adjustment and optimization of welding trajectory can be achieved, the accuracy of welding trajectory can be improved, and the problem that the preset welding trajectory of the robot is difficult to adapt to the geometric deviation of the workpiece and the change of weld position can be solved, providing technical support for the widespread application of robot welding technology.

[0174] Since the processing and functions implemented by the device of this embodiment basically correspond to the embodiments, principles and examples of the aforementioned method, for any details not fully described in this embodiment, please refer to the relevant descriptions in the aforementioned embodiments and will not be repeated here.

[0175] The technical solution of this invention inputs workpiece point cloud data into a point cloud semantic segmentation model to obtain weld point cloud data for the workpiece's linear welds. This weld point cloud data is then fitted to obtain the weld points for the linear welds. Finally, the weld points are fine-tuned and their poses estimated to obtain the weld trajectory for the linear welds. This not only enables precise identification and extraction of linear weld point clouds, improving the efficiency and accuracy of weld detection, but also enhances the precision and adaptability of welding trajectory planning.

[0176] According to an embodiment of the present invention, a welding robot corresponding to the welding trajectory planning device of the welding robot is further provided. The welding robot may include: the welding trajectory planning device of the welding robot described above.

[0177] Since the processing and functions implemented by the welding robot of this embodiment basically correspond to the embodiments, principles and examples of the aforementioned device, for any details not fully described in this embodiment, please refer to the relevant descriptions in the aforementioned embodiments and will not be repeated here.

[0178] The technical solution of this invention inputs workpiece point cloud data into a point cloud semantic segmentation model to obtain weld point cloud data for the workpiece's linear welds. This weld point cloud data is then fitted to obtain the weld points for the linear welds. Finally, the weld points are fine-tuned and their poses estimated to obtain the weld trajectory for the linear welds. This not only enables precise identification and extraction of linear weld point clouds, improving the efficiency and accuracy of weld detection, but also enhances the precision and adaptability of welding trajectory planning.

[0179] According to an embodiment of the present invention, a storage medium corresponding to a welding trajectory planning method for a welding robot is also provided, wherein the storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the welding trajectory planning method for the welding robot described above.

[0180] Since the processing and functions implemented by the storage medium of this embodiment basically correspond to the embodiments, principles and examples of the aforementioned method, for any details not fully described in this embodiment, please refer to the relevant descriptions in the aforementioned embodiments and will not be repeated here.

[0181] The technical solution of this invention inputs workpiece point cloud data into a point cloud semantic segmentation model to obtain weld point cloud data for the workpiece's linear welds. This weld point cloud data is then fitted to obtain the weld points for the linear welds. Finally, the weld points are fine-tuned and their poses estimated to obtain the weld trajectory for the linear welds. This not only enables precise identification and extraction of linear weld point clouds, improving the efficiency and accuracy of weld detection, but also enhances the precision and adaptability of welding trajectory planning.

[0182] According to an embodiment of the present invention, a computer program product corresponding to the welding trajectory planning method of a welding robot is also provided. The computer program product includes a computer program, which implements the steps of the welding trajectory planning method of the welding robot when processed and executed.

[0183] Since the processing and functions implemented by the computer program product of this embodiment basically correspond to the embodiments, principles and examples of the aforementioned method, for any details not fully described in this embodiment, please refer to the relevant descriptions in the aforementioned embodiments and will not be repeated here.

[0184] The technical solution of this invention inputs workpiece point cloud data into a point cloud semantic segmentation model to obtain weld point cloud data for the workpiece's linear welds. This weld point cloud data is then fitted to obtain the weld points for the linear welds. Finally, the weld points are fine-tuned and their poses estimated to obtain the weld trajectory for the linear welds. This not only enables precise identification and extraction of linear weld point clouds, improving the efficiency and accuracy of weld detection, but also enhances the precision and adaptability of welding trajectory planning.

[0185] In summary, it is easy for those skilled in the art to understand that, under the premise of no conflict, the above-mentioned advantageous methods can be freely combined and superimposed.

[0186] The foregoing description is merely an embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of the claims.

Claims

1. A welding trajectory planning method for a welding robot, characterized in that: include: Obtain point cloud data of the workpiece; Inputting the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain weld point cloud data of the linear weld of the workpiece; The preset point cloud semantic segmentation model is used to determine the weld point cloud data of the straight weld of the workpiece according to the global topological features and global geometric features of the workpiece point cloud data; Fitting the weld point cloud data to obtain weld points of the linear weld; Performing welding point fine-tuning and posture estimation on the welding points of the weld seam to obtain a welding trajectory of the linear weld seam; The step of fitting the weld point cloud data to obtain the weld points of the linear weld includes: Segmenting the weld point cloud data using density clustering with normal vector constraints to obtain a weld point cloud set; the weld point cloud set includes point cloud clusters; Performing straight line fitting on each point cloud cluster in the weld point cloud set to obtain a fitting straight line for each point cloud cluster; All points in each point cloud cluster are projected onto the corresponding fitting line, and coordinate points are evenly sampled at preset intervals within the projection range to generate weld points of the linear weld.

2. The welding trajectory planning method of the welding robot according to claim 1, characterized in that: The point cloud semantic segmentation model includes: a topological feature learning module, a geometric feature learning module, a feature fusion module, a label prediction module, and a point cloud extraction module; Inputting the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain weld point cloud data of the linear weld of the workpiece, including: Inputting the point cloud data of the workpiece into the topological feature learning module to obtain global topological features; Inputting the point cloud data of the workpiece into the geometric feature learning module to obtain global geometric features; Inputting the global topological features and the global geometric features into the feature fusion module to obtain fused features; The fusion feature is processed by the label prediction module to obtain the probability that each point cloud data in the point cloud data of the workpiece belongs to the point cloud data of the linear weld; The point cloud extraction module extracts the weld point cloud data of the complete straight weld according to the probability that each point cloud data belongs to the point cloud data of the straight weld.

3. The welding trajectory planning method of the welding robot according to claim 1, characterized in that: The weld point cloud data is segmented using density clustering with normal vector constraints to obtain a weld point cloud set, including: Determining whether a point cloud cluster in the weld point cloud data satisfies a preset first condition and a preset second condition; If the point cloud cluster meets the preset first condition and the preset second condition, then adding the point cloud cluster to the weld point cloud set; If the point cloud cluster does not meet the preset first condition or the preset second condition, the point cloud cluster is not added to the weld point cloud set; Among them, the preset first condition is that the number of point cloud data within the preset first range of the point cloud cluster is greater than the point cloud data number threshold; the preset second condition is that the angle between the normal vectors of all point cloud data within the preset first range of the point cloud cluster is less than or equal to the angle threshold.

4. The welding trajectory planning method of the welding robot according to claim 1, characterized in that: Fine-tuning the weld points and estimating their position and posture to obtain a welding trajectory of the linear weld includes: For each weld point in the weld seam, recording a set of point cloud data of all weld points within a preset second range of the current weld point as a local point cloud set; Determining whether planes whose number is greater than or equal to a plane number threshold can be segmented from the local point cloud set; If the number of planes greater than or equal to the plane number threshold cannot be segmented from the local point cloud set, no weld fine-tuning is performed on the current weld; If a number of planes greater than or equal to a plane number threshold can be segmented from the local point cloud set, then an intersection line of the segmented planes is calculated; the current weld point is projected onto the intersection line to obtain a projection point; and the projection point is used as the weld point after the weld point is fine-tuned. The pose of all welds that have completed fine-tuning is estimated to obtain the welding trajectory of the straight weld.

5. The welding trajectory planning method of the welding robot according to claim 4, characterized in that: The pose of all welds that have completed fine-tuning is estimated to obtain the welding trajectory of the linear weld, including: For each solder point after solder point fine-tuning, normalize the direction vector of the intersection line of the current solder point to obtain an x-axis basis vector; In the robot base coordinate system, a point on one side of the weld of the workpiece is used as a viewpoint, and the centroid of the point set of the neighborhood point cloud of the current weld point projected onto the intersection line is calculated to obtain a direction vector pointing from the viewpoint to the centroid, which is recorded as a first direction vector. Calculating a normal vector set of the plane, and determining whether there is a normal vector in the normal vector set that forms an obtuse angle with the first direction vector; If there is a normal vector whose angle with the first direction vector is an obtuse angle, then the direction of the normal vector whose angle with the first direction vector is an obtuse angle is negated; If there is no normal vector whose angle with the first direction vector is an obtuse angle, summing all normal vectors in the normal vector set to obtain a second direction vector; calculating the component of the second direction vector perpendicular to the x-axis basis vector, and normalizing it to obtain a z-axis basis vector; Determine a y-axis basis vector based on the x-axis basis vector and the z-axis basis vector; The coordinates of each weld point in the weld seam are used as the origin of the coordinate system, and the x-axis basis vector, the z-axis basis vector, and the y-axis basis vector constitute the coordinate system orientation to form the welding trajectory of the straight weld seam.

6. A welding trajectory planning device for a welding robot, characterized in that: include: an acquisition unit configured to acquire point cloud data of a workpiece; a semantic segmentation unit configured to input the point cloud data of the workpiece into a preset point cloud semantic segmentation model to obtain weld point cloud data of a linear weld of the workpiece; a fitting unit configured to fit the weld point cloud data to obtain weld points of the linear weld; A trajectory planning unit is configured to perform weld point fine-tuning and pose estimation on the weld points of the weld seam to obtain a welding trajectory of the straight weld seam; The fitting unit fits the weld point cloud data to obtain the weld points of the linear weld, including: Segmenting the weld point cloud data using density clustering with normal vector constraints to obtain a weld point cloud set; the weld point cloud set includes point cloud clusters; Performing straight line fitting on each point cloud cluster in the weld point cloud set to obtain a fitting straight line for each point cloud cluster; All points in each point cloud cluster are projected onto the corresponding fitting line, and coordinate points are evenly sampled at preset intervals within the projection range to generate weld points of the linear weld.

7. A welding robot, characterized in that: include: The welding trajectory planning device for a welding robot as claimed in claim 6.

8. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the welding trajectory planning method for the welding robot according to any one of claims 1 to 5.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images