Steel pipe welding robot control method and system based on machine vision

Abstract

The invention discloses a steel pipe welding robot control method and system based on machine vision, and relates to the technical field of image processing. The method comprises the steps of collecting point cloud data of a welding seam area of a steel pipe and a to-be-welded object, conducting segmentation processing on the point cloud data, outputting point cloud data of all the areas, recognizing a targeted analysis domain of welding seam point cloud through a preset algorithm, screening distance feature points and outputting a welding seam feature point set, and constructing an interpolation curve through an interpolation function. Welding path coordinates and welding gun tail end pose data are calculated, a welding instruction is generated, and a robot is controlled to conduct welding operation; according to the method, the tedious process of debugging the welding gun postures one by one in point position is reduced, the matching degree of the welding gun postures at different point positions and the welding seam angle is guaranteed, posture errors caused by manual operation are avoided, welding gun posture control is more consistent and accurate, and robot welding is conveniently controlled.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a control method and system for a steel pipe welding robot based on machine vision. Background Technology

[0002] Robotic welding is an automated technology that utilizes industrial robots to complete welding operations. It is widely used in automotive manufacturing, shipbuilding, and construction machinery, and is a key component of modern intelligent manufacturing and industrial automation upgrades. By precisely controlling the welding torch trajectory, speed, and parameters through pre-programmed instructions, it achieves high-quality, high-efficiency continuous production. Compared to manual welding, robotic welding offers advantages such as high stability, high precision, and wide environmental adaptability. It can operate continuously in harsh environments such as high temperatures and dusty conditions, significantly reducing labor intensity and improving product consistency. Modern intelligent welding robots can also incorporate sensing technology to achieve automatic weld seam tracking and real-time parameter adjustment.

[0003] Publication No. CN115032944A discloses a laser welding control method based on machine vision, which mainly includes: acquiring a surface image of the weld to be welded, and performing grayscale conversion and edge detection on the surface image to obtain an edge image; determining an initial welding path based on the center line of the weld region in the edge image, and performing piecewise linear fitting based on least squares on the initial welding path to obtain an initial fitted path; obtaining the fitting evaluation value of each line segment on the initial fitted path; performing multiple linear regression curve fitting on the adjacent line segments in the initial fitted path whose fitting evaluation values ​​are less than a preset threshold to obtain a target fitted path; and performing laser welding on the weld to be welded using the target fitted path.

[0004] The above method obtains the target fitting path by fitting the surface image of the weld seam for welding. This method works well when welding flat objects, but when welding V-shaped weld seams, the fitted multiple linear regression curve is not accurate enough. Moreover, relying on only a single fitting path lacks the constraint of the welding torch position on the machine, resulting in poor welding effect on the object to be welded, causing economic and time losses. Summary of the Invention

[0005] The purpose of this invention is to solve the problems of insufficient accuracy caused by the reliance on 2D information for weld feature extraction in traditional welding robots and the fact that existing path generation methods are only suitable for simple bevel welds. Therefore, this invention proposes a control method and system for steel pipe welding robots based on machine vision.

[0006] In a first aspect of this invention, a control method for a steel pipe welding robot based on machine vision is first proposed, the method comprising: Point cloud data of the weld seam area between the steel pipe and the object to be welded is collected, and the point cloud data is segmented and output as point cloud data of each region; the region includes planar region and curved surface region; The target analysis domain of the weld point cloud is identified by the preset algorithm for the point cloud data of each region, and the weld feature point set is output by filtering the distance feature points. The interpolation curve of the weld feature point set is constructed according to the interpolation function. The welding path coordinates are calculated based on the interpolation curve, and the position data of the welding gun end at each point on the welding path coordinates are obtained by calculating the welding path coordinates. Welding instructions are generated using the welding path and the pose data, and the robot is controlled to perform welding operations on the object to be welded according to the welding instructions.

[0007] Optionally, the point cloud data is segmented to output point cloud data for each region, including: Pre-set the maximum number of iterations, the distance threshold from a point to the plane, the threshold for the number of interior points, the threshold for the cluster radius, and the threshold for the number of neighborhood points; In each iteration, three non-collinear points are randomly selected to fit the plane equation as the first plane region. The distance from all points to the first plane region is calculated. All distances are traversed, and points with a distance less than the distance threshold are determined as interior points. If the number of interior points is greater than the number threshold, the first plane region is retained and all interior points are extracted as a plane region. Repeat the iteration until all regions that can be fitted to a plane are segmented, and remove the segmented interior points from the original point cloud data to obtain the residual point cloud that has not been fitted to a plane. Traverse each point p in the residual point cloud, count the number of points in the neighborhood centered at point p with a clustering radius threshold as the radius. If the number of points in the neighborhood is greater than the neighborhood point count threshold, then all points in the neighborhood are grouped into the same cluster, and the operation is repeated until all residual points are clustered and the segmented point cloud data of each region is output.

[0008] Optionally, the interpolation curves of the weld feature points are constructed based on the interpolation function to output the weld feature points, including: The point cloud data of each region is filtered by height and normal vector. If the height difference between the point cloud and the substrate plane in the weld seam area of ​​the medium and thick plate is lower than the height difference threshold, and the angle between the normal vector and the substrate plane is within the angle threshold, then the weld seam candidate area is selected and expanded into the target analysis domain of the point cloud. Two adjacent planar regions within the target analysis domain are selected and defined as the second planar region and the third planar region, respectively. The distance from each point in the third planar region to the second planar region is calculated. Points with a distance less than the distance threshold are taken as weld feature points. Weld feature points that meet the conditions are collected to obtain the weld feature point set. The interpolation curve of the weld feature points is obtained by fitting each feature point in the weld feature point set.

[0009] Optionally, the welding path coordinates are calculated based on the interpolation curve, including: Obtain the coordinates of the feature points of the weld in the interpolation curve, the preset weld thickness, and the weld width; the feature points of the weld include the two upper endpoints A and B of the V-shaped weld, the bottom endpoint C of the V-shaped weld, and the point D, the midpoint between endpoints A and C, and the point E, the midpoint between endpoints A and B; Substituting these values ​​into the formula, the welding path coordinates of the weld are calculated as follows:

[0010] in, For the number of welding layers, The width of the m-th layer cross section. This refers to the number of weld passes. Let A, B, C, D, and E represent the depth axis coordinates of feature points A, B, C, D, and E on the i-th V-section of the weld, respectively; n is the number of feature points, and h is the preset weld thickness. The optimized weld bead thickness is given by w, which is the preset weld bead width. These represent the transverse axis coordinates of feature points A, B, D, and E on the i-th V-section of the weld, respectively. After optimizing the number of welding layers, number of weld passes, weld pass thickness, and weld pass width, the coordinates of each weld pass path are calculated.

[0011] Optionally, the pose data of the welding torch tip at each point on the welding path coordinates is calculated using the welding path coordinates, including: Weighted neighborhood point cloud data is obtained by assigning weights to the neighborhood point cloud of each point on the welding path coordinates. After decentralizing the weighted neighborhood point cloud data, a weighted covariance matrix is ​​constructed. Singular value decomposition is performed on the weighted covariance matrix, and the eigenvector corresponding to the smallest eigenvalue is taken as the normal vector of the target point. The normal vector of the target point is then used as the pose data. The target point is any point on the welding path coordinates.

[0012] In a second aspect of this invention, a machine vision-based steel pipe welding robot control system is proposed. The system includes a segmentation module, a construction module, a path calculation module, and a welding control module, wherein: The segmentation module is used to collect point cloud data of the weld seam area between the steel pipe and the object to be welded, and to segment the point cloud data to output point cloud data of each region; the region includes planar region and curved surface region. The construction module is used to identify the target analysis domain of the weld point cloud from the point cloud data of each region through a preset algorithm, and to filter the distance feature points to output the weld feature point set. The interpolation curve of the weld feature point set is constructed and output according to the interpolation function. The path calculation module is used to calculate the welding path coordinates based on the interpolation curve, and to obtain the pose data of the welding gun end at each point on the welding path coordinates through the welding path coordinates. The welding control module is used to generate welding instructions based on the welding path and the pose data, and to control the robot to perform welding operations on the object to be welded according to the welding instructions.

[0013] Optionally, the segmentation module includes an interior point determination module, a removal module, and a region division module, wherein: Pre-set the maximum number of iterations, the distance threshold from a point to the plane, the threshold for the number of interior points, the threshold for the cluster radius, and the threshold for the number of neighborhood points; The interior point determination module is used to randomly select 3 non-collinear points in each iteration to fit the plane equation as the first plane region, calculate the distance from all points to the first plane region, traverse all distances, and determine the points whose distance is less than the distance threshold as interior points. If the number of interior points is greater than the number threshold, the first plane region is retained and all interior points are extracted as a plane region. The removal module is used to iterate repeatedly until all regions that can be fitted to a plane are segmented, and at the same time remove the segmented interior points from the original point cloud data to obtain the residual point cloud that has not been fitted to a plane. The region segmentation module is used to traverse each point p in the residual point cloud, count the number of points in the neighborhood centered on point p and with a clustering radius threshold as the radius. If the number of points in the neighborhood is greater than the neighborhood point count threshold, then all points in the neighborhood are grouped into the same cluster, and the operation is repeated until all residual points are clustered and the segmented point cloud data of each region is output.

[0014] Optionally, the construction module includes a filtering module, a calculation module, and a fitting module, wherein: The filtering module is used to filter the point cloud data of each region by height and normal vector. If the height difference between the point cloud of the weld seam region of the medium and thick plate and the substrate plane is lower than the height difference threshold, and the angle between the normal vector and the substrate plane is within the angle threshold, then the weld seam candidate region is filtered out and expanded into the target analysis domain of the point cloud. The calculation module is used to select two adjacent planar regions within the target analysis domain and define them as the second planar region and the third planar region, respectively. It calculates the distance from each point in the third planar region to the second planar region, takes points whose distance is less than the distance threshold as weld feature points, and collects weld feature points that meet the conditions to obtain a weld feature point set. The fitting module is used to fit each feature point in the weld feature point set to obtain the weld feature point interpolation curve.

[0015] Optionally, the path calculation module includes a parameter calculation module and a coordinate calculation module, wherein: Obtain the coordinates of the feature points of the weld in the interpolation curve, the preset weld thickness, and the weld width; the feature points of the weld include the two upper endpoints A and B of the V-shaped weld, the bottom endpoint C of the V-shaped weld, and the point D, the midpoint between endpoints A and C, and the point E, the midpoint between endpoints A and B; The parameter calculation module is used to substitute these parameters into a formula to calculate the welding path coordinates of the weld. The process is as follows:

[0016] in, For the number of welding layers, The width of the m-th layer cross section. This refers to the number of weld passes. Let A, B, C, D, and E represent the depth axis coordinates of feature points A, B, C, D, and E on the i-th V-section of the weld, respectively; n is the number of feature points, and h is the preset weld thickness. The optimized weld bead thickness is given by w, which is the preset weld bead width. These represent the transverse axis coordinates of feature points A, B, D, and E on the i-th V-section of the weld, respectively. The coordinate calculation module is used to optimize the number of welding layers, the number of weld passes, the weld pass thickness, and the weld pass width, and then calculate the coordinates of each welding path.

[0017] Optionally, the welding control module includes a matrix construction module and a pose generation module, wherein: The matrix construction module is used to assign weights to the neighborhood point cloud of each point on the welding path coordinate to obtain weighted neighborhood point cloud data, and to construct a weighted covariance matrix after decentralizing the weighted neighborhood point cloud data. The pose generation module is used to perform singular value decomposition on the weighted covariance matrix, take the eigenvector corresponding to the smallest eigenvalue as the normal vector of the target point, and use the normal vector of the target point as pose data; the target point is any point on the welding path coordinates.

[0018] The beneficial effects of this invention are: This invention proposes a machine vision-based control method for steel pipe welding robots. By collecting point cloud data of the weld area, and then segmenting the region to lock the weld target analysis domain, combined with distance feature point screening and interpolation curve modeling, the actual spatial shape of the weld is accurately restored. The welding path coordinates calculated in this way can closely match the real weld trajectory. Based on the weld interpolation curve, the welding torch end pose data of each point on the welding path can be accurately calculated, reducing the tedious process of adjusting the welding torch posture at each point. This solves the problems of insufficient accuracy caused by the reliance on 2D information for weld feature extraction in traditional welding robots and the fact that existing path generation methods are only suitable for simple bevel welds. It ensures the matching degree between the welding torch posture and the weld angle at different points, avoids posture errors caused by manual operation, and makes the welding torch posture control more consistent and accurate, which is conducive to controlling robot welding. Attached Figure Description

[0019] The invention will now be further described with reference to the accompanying drawings.

[0020] Figure 1 A flowchart illustrating a machine vision-based control method for a steel pipe welding robot, provided as an embodiment of the present invention; Figure 2 This is a framework diagram of a steel pipe welding robot control system based on machine vision, provided for an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and B can represent: A alone, A and B simultaneously, and B alone. Furthermore, descriptions involving "first," "second," etc., in this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" can explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0022] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] This invention provides a machine vision-based control method for a steel pipe welding robot. See also... Figure 1 , Figure 1 A flowchart illustrating a machine vision-based control method for a steel pipe welding robot, provided as an embodiment of the present invention. The method includes the following steps: S101: Collect point cloud data of the weld area between the steel pipe and the object to be welded, segment the point cloud data and output the point cloud data of each area. S102, the target analysis domain of the weld seam point cloud is identified by the preset algorithm for the point cloud data of each region, and the weld seam feature point set is output by filtering the distance feature points. The interpolation curve of the weld seam feature point set is constructed according to the interpolation function. S103, calculate the welding path coordinates based on the interpolation curve, and obtain the pose data of the welding gun end at each point on the welding path coordinates through the welding path coordinates. S104 generates welding instructions based on welding path and pose data, and controls the robot to perform welding operations on the object to be welded according to the welding instructions.

[0024] The region includes planar regions and curved surface regions.

[0025] This invention provides a machine vision-based control method for steel pipe welding robots. By collecting point cloud data of the weld seam area of ​​the steel pipe and the object to be welded, and then locking the weld seam target analysis domain through region segmentation, combined with distance feature point screening and interpolation curve modeling, the actual spatial shape of the weld seam is accurately restored. The welding path coordinates calculated in this way can closely match the actual weld seam trajectory. Based on the weld seam interpolation curve, the welding torch end pose data of each point on the welding path can be accurately calculated, reducing the tedious process of adjusting the welding torch posture at each point, ensuring the matching degree between the welding torch posture and the weld seam angle at different points, avoiding posture errors caused by manual operation, and making the welding torch posture control more consistent and accurate, which is convenient for controlling robot welding.

[0026] In one implementation, an image of the weld seam area of ​​the object to be welded is acquired by a ToF camera mounted on a robot. The ToF camera emits modulated light and receives the reflected signal. The three-dimensional coordinates of each measurement point are calculated using the time of flight of light, and the original point cloud dataset containing a large number of spatial points is output. The targeted analysis domain of the weld seam point cloud is essentially a local three-dimensional spatial region around the weld seam, defined based on the appearance features of the welded workpiece.

[0027] In one implementation, the original point cloud data of the weld area of ​​the object to be welded is preprocessed to obtain point cloud data. The process is as follows: The radius filtering method is used to traverse each point in the point cloud, set a neighborhood range with a fixed radius centered on the point, count the number of points in the neighborhood, and if the number of points is lower than the preset threshold, it is identified as an outlier noise point and removed, thus obtaining a preliminary purified point cloud with obvious noise removed, eliminating invalid data introduced by insufficient accuracy of the ToF camera and ambient light. Construct a voxel mesh to divide the 3D point cloud space into cubic meshes of equal volume; By sampling all points within each grid and retaining only one representative point (usually the grid center point or centroid point), point cloud data is obtained, which significantly reduces the amount of data while preserving the spatial structural features of the weld.

[0028] In one embodiment, segmenting point cloud data and outputting point cloud data for each region includes: Pre-set the maximum number of iterations, the distance threshold from a point to the plane, the threshold for the number of interior points, the threshold for the cluster radius, and the threshold for the number of neighborhood points; In each iteration, three non-collinear points are randomly selected to fit the plane equation as the first plane region. The distance from all points to the first plane region is calculated. All distances are traversed, and points with a distance less than the distance threshold are determined as interior points. If the number of interior points is greater than the number threshold, the first plane region is retained and all interior points are extracted as a plane region. Repeat the iteration until all regions that can be fitted to a plane are segmented, and remove the segmented interior points from the original point cloud data to obtain the residual point cloud that has not been fitted to a plane. Traverse each point p in the residual point cloud, count the number of points in the neighborhood centered at point p with a clustering radius threshold as the radius. If the number of points in the neighborhood is greater than the neighborhood point count threshold, then all points in the neighborhood are grouped into the same cluster, and the operation is repeated until all residual points are clustered and the segmented point cloud data of each region is output.

[0029] In one implementation, regular planar regions such as the substrate plane and V-groove slope are accurately identified and segmented from the preprocessed weld point cloud. At the same time, the mathematical equations (ax+by+cz+l=0) of each plane are fitted to define the basic planar regions for subsequent weld feature point extraction. For the residual point cloud that cannot be classified into planes after plane fitting, the complex regions are accurately clustered and segmented by statistically analyzing the neighborhood point density, thus overcoming the limitation of traditional methods that can only handle planar structures.

[0030] In one implementation, the point cloud data of each region is divided by this method, which retains the advantages of traditional algorithms such as strong noise resistance and high segmentation efficiency, and achieves full coverage and accurate segmentation of the planar region. This provides more accurate regional division for subsequent extraction of weld feature points, improves the basic data quality of the entire welding path generation, and can adapt to multiple types of medium and thick plate welds such as fillet welds, lap welds, and butt welds. Compared with existing single-function segmentation technologies, it has stronger scene adaptability.

[0031] In one embodiment, constructing an interpolation curve for the weld feature points based on the interpolation function includes: The point cloud data of each region is filtered by height and normal vector. If the height difference between the point cloud and the substrate plane in the weld seam area of ​​the medium and thick plate is lower than the height difference threshold, and the angle between the normal vector and the substrate plane is within the angle threshold, then the weld seam candidate area is selected and expanded into the target analysis domain of the point cloud. Two adjacent planar regions within the target analysis domain are selected and defined as the second planar region and the third planar region, respectively. The distance from each point in the third planar region to the second planar region is calculated. Points with a distance less than the distance threshold are taken as weld feature points. Weld feature points that meet the conditions are collected to obtain the weld feature point set. The interpolation curve of the weld feature points is obtained by fitting each feature point in the weld feature point set.

[0032] In one implementation, the preprocessed initial point cloud data is spatially clipped by targeted analysis domain extraction, defining the weld and its directly related surrounding areas as the core analysis scope, and eliminating redundant point clouds such as those on the far-end substrate, significantly reducing the computational load of subsequent distance calculations. Based on the distance method, weld feature points at the intersection of different planes (such as the edge of a V-groove) are selected, and the integrity and noise resistance of the feature points are balanced by a threshold set by experimental data. These feature points serve as the core characterization data of the weld's three-dimensional geometric contour, providing a key geometric benchmark for the trajectory smoothing of subsequent feature point interpolation and the accurate planning of multi-layer and multi-pass welding paths.

[0033] In one implementation, the formula is called: Calculate the distances from points in the third planar region to the second planar region; where a, b, c, and l are the coefficients of the plane equation. The three-dimensional coordinates of the point in the point cloud; The interpolation curve of the weld feature points is obtained by fitting each feature point in the weld feature point set. The process is as follows: Construct a cubic spline interpolation function S(x) that satisfies And within the interval The expression inside is a cubic polynomial; Emmett interpolation Substitute into the basis functions The expression is obtained by solving the first derivative of the interpolation function using the chasing method. To ensure the continuity of the first and second derivatives of the curve in, Represents the coordinates of the feature point; express The first derivative at that point; For Hermitian interpolation basis functions.

[0034] In one implementation, the role of interpolation curve construction is: First, the trajectory completion and smoothing process is carried out on the feature points of discrete and sparse welds. By constructing a cubic spline function that satisfies the continuity of the first and second derivatives, the gaps between feature points and trajectory breakpoints are eliminated, and the stuttering or trajectory change during the movement of the welding torch is avoided. Secondly, in response to the requirements of the swaying welding process for V-shaped butt welds, the number of feature points on the left and right sides of the weld is unified by interpolation to ensure the symmetry of the swaying welding action and reduce weld formation defects caused by unilateral offset. Third, thanks to its excellent local controllability, deviations at a single feature point only affect the curve shape of adjacent intervals and do not spread to the entire region, thus ensuring the accuracy and stability of the overall path.

[0035] Compared to the shortcomings of linear interpolation, which is prone to trajectory inflection points, and quadratic interpolation, which lacks smoothness, cubic spline interpolation ensures that the curve passes through all feature point nodes and achieves a smooth transition of curvature across the entire domain, effectively improving the smoothness of the welding torch movement and the quality of weld formation.

[0036] In one embodiment, calculating the welding path coordinates based on the interpolation curve includes: Obtain the coordinates of the feature points of the weld in the interpolation curve, the preset weld thickness and weld width; the feature points of the weld include the two upper endpoints A and B of the V-shaped weld, the bottom endpoint C of the V-shaped weld, and the point D at the midpoint between endpoints A and C and the point E at the midpoint between endpoints A and B; Substituting these values ​​into the formula, the welding path coordinates of the weld are calculated as follows:

[0037] in, For the number of welding layers, The width of the m-th layer cross section. This refers to the number of weld passes. Let A, B, C, D, and E represent the depth axis coordinates of feature points A, B, C, D, and E on the i-th V-section of the weld, respectively; n is the number of feature points, and h is the preset weld thickness. The optimized weld bead thickness is given by w, which is the preset weld bead width. These represent the transverse axis coordinates of feature points A, B, D, and E on the i-th V-section of the weld, respectively. After optimizing the number of welding layers, number of weld passes, weld pass thickness, and weld pass width, the coordinates of each weld pass path are calculated.

[0038] In one implementation, the coordinates of each weld path are calculated as follows:

[0039] in, This represents the coordinates of the path of the j-th weld bead in the i-th layer. This represents the mean of the horizontal axis coordinates of feature points B and E; This represents the mean depth axis coordinates of feature points B and E. This represents the actual number of weld layers. For the optimized weld bead thickness, This represents the actual number of weld passes. For the adjusted weld width, For the error threshold, The integer symbol.

[0040] In one implementation, based on the smooth weld feature point trajectory, the number of welding layers and the number of weld passes per layer are calculated according to the structural characteristics and process requirements of the fillet weld (V-groove) weld, and the thickness and width of the weld passes are adaptively adjusted to generate a precise three-dimensional coordinate set for each layer and each weld pass. At the same time, the oscillation welding trajectory is planned for the V-groove weld, and the bottom-up path is designed for the vertical plate weld to solve the problem of molten droplet falling. This provides a complete position and movement reference for the welding torch and lays the foundation for subsequent welding torch attitude calculation. One implementation overcomes the limitation of most existing MLMP path generation methods that are only applicable to V-groove welds, achieving full coverage of common industrial medium and thick plate welds. Compared with the drawbacks of traditional methods that require manual intervention, are time-consuming, and prone to errors, this method relies on three-dimensional point cloud geometric features to automatically generate paths, significantly shortening path planning time and reducing path errors. The process adaptability and scenario universality improve the automation level and weld formation quality of multi-layer and multi-pass welding.

[0041] In one embodiment, the pose data of the welding torch tip at each point on the welding path coordinates is calculated using the welding path coordinates, including: Weighted neighborhood point cloud data is obtained by assigning weights to the neighborhood point cloud of each point on the welding path coordinates. After decentralizing the weighted neighborhood point cloud data, a weighted covariance matrix is ​​constructed. Singular value decomposition is performed on the weighted covariance matrix, and the eigenvector corresponding to the smallest eigenvalue is taken as the normal vector of the target point. The normal vector of the target point is then used as the pose data. The target point is any point on the welding path coordinate system.

[0042] In one implementation, weighted neighborhood point cloud data is obtained by assigning weights to the neighborhood point clouds of each point on the welding path coordinates. The process is as follows: By calculating each point in the neighborhood Calculate the Euclidean distance to the center point c and substitute it into the formula:

[0043] in, Representing neighborhood points The weight, For neighborhood points The Euclidean distance to the center c. The average distance from the neighboring points to the center point. This is the Euclidean distance threshold.

[0044] In one implementation, the normal vector is obtained by weighted analysis of the neighborhood point cloud of each point on the welding path. The normal vector is then transformed into the tilt angle and attitude parameters of the welding torch tip. This completes the defect that the path coordinates can only define the position of the welding torch, forming a complete welding control command of "position + attitude". At the same time, the interference of noise points and non-planar points in the neighborhood on attitude estimation is reduced by the weighting strategy, ensuring that the welding torch is perpendicular to the local surface of the weld at each point, ensuring a stable wire feed angle and uniform arc shape, thereby improving the weld formation quality.

[0045] In one implementation, compared to the drawbacks of traditional PCA normal vector estimation, which assigns equal weights to neighboring points and is susceptible to attitude deviations caused by distant noise and bevel junction interference, weighted PCA improves the accuracy and robustness of attitude estimation by assigning high weights to nearby points and low weights to distant points. Compared to the limitations of the traditional mode, which has a fixed welding torch attitude and cannot adapt to changes in weld surface morphology, it can dynamically generate adaptive attitudes for each point, is compatible with various fillet welds, and its attitude calculation is automatically completed based on point cloud geometric features without manual intervention, which greatly shortens the attitude calibration time and ensures the consistency of attitude estimation.

[0046] Based on the same inventive concept, this invention also provides a machine vision-based control system for a steel pipe welding robot. See also... Figure 2 , Figure 2 A framework diagram of a machine vision-based steel pipe welding robot control system provided in this embodiment of the invention is shown. The system includes a segmentation module, a construction module, a path calculation module, and a welding control module, wherein: The segmentation module is used to collect point cloud data of the weld seam area between the steel pipe and the object to be welded, segment the point cloud data, and output point cloud data of each region; the regions include planar regions and curved surface regions. The module is used to identify the target analysis domain of the weld point cloud from the point cloud data of each region through a preset algorithm, and to filter the distance feature points to output the weld feature point set. Based on the interpolation function, the interpolation curve of the weld feature point set is constructed and output. The path calculation module is used to calculate the welding path coordinates based on the interpolation curve, and to obtain the pose data of the welding gun end at each point on the welding path coordinates. The welding control module is used to generate welding instructions based on welding path and pose data, and to control the robot to perform welding operations on the object to be welded according to the welding instructions.

[0047] The steel pipe welding robot control system based on machine vision provided by this invention collects point cloud data of the weld seam area of ​​the steel pipe and the object to be welded, then locks the weld seam target analysis domain through region segmentation, and combines distance feature point screening and interpolation curve modeling to accurately restore the actual spatial shape of the weld seam. The welding path coordinates calculated in this way can closely match the actual weld seam trajectory. According to the weld seam interpolation curve, the welding gun end pose data of each point on the welding path can be accurately calculated, reducing the tedious process of adjusting the welding gun posture at each point, ensuring the matching degree between the welding gun posture and the weld seam angle at different points, avoiding posture errors caused by manual operation, and making the welding gun posture control more consistent and accurate, which is convenient for controlling robot welding.

[0048] In one embodiment, the segmentation module includes an interior point determination module, a removal module, and a region division module, wherein: Pre-set the maximum number of iterations, the distance threshold from a point to the plane, the threshold for the number of interior points, the threshold for the cluster radius, and the threshold for the number of neighborhood points; The interior point determination module is used to randomly select 3 non-collinear points in each iteration to fit the plane equation as the first plane region, calculate the distance from all points to the first plane region, traverse all distances, and determine the points whose distance is less than the distance threshold as interior points. If the number of interior points is greater than the number threshold, the first plane region is retained and all interior points are extracted as a plane region. The removal module is used to iterate repeatedly until all regions that can be fitted to a plane are segmented, while removing the segmented interior points from the original point cloud data to obtain the residual point cloud that is not fitted to a plane. The region segmentation module is used to traverse each point p in the residual point cloud, count the number of points in the neighborhood centered at point p with a clustering radius threshold as the radius, and if the number of points in the neighborhood is greater than the neighborhood point count threshold, then all points in the neighborhood are grouped into the same cluster, and the operation is repeated until all residual points are clustered and the segmented point cloud data of each region is output.

[0049] In one embodiment, the construction module includes a filtering module, a calculation module, and a fitting module, wherein: The filtering module is used to filter the point cloud data of each region by height and normal vector. If the height difference between the point cloud of the weld seam region of the medium and thick plate and the substrate plane is lower than the height difference threshold, and the angle between the normal vector and the substrate plane is within the angle threshold, then the weld seam candidate region is filtered out and expanded into the target analysis domain of the point cloud. The calculation module is used to select two adjacent planar regions within the target analysis domain and define them as the second planar region and the third planar region, respectively. It calculates the distance from each point in the third planar region to the second planar region, and takes the points whose distance is less than the distance threshold as weld feature points. It collects weld feature points that meet the conditions to obtain a weld feature point set. The fitting module is used to fit each feature point in the weld feature point set to obtain the weld feature point interpolation curve.

[0050] In one embodiment, the path calculation module includes a parameter calculation module and a coordinate calculation module, wherein: Obtain the coordinates of the feature points of the weld in the interpolation curve, the preset weld thickness and weld width; the feature points of the weld include the two upper endpoints A and B of the V-shaped weld, the bottom endpoint C of the V-shaped weld, and the point D at the midpoint between endpoints A and C and the point E at the midpoint between endpoints A and B; The parameter calculation module is used to substitute these parameters into the formula to calculate the welding path coordinates of the weld. The process is as follows:

[0051] in, For the number of welding layers, The width of the m-th layer cross section. This refers to the number of weld passes. Let A, B, C, D, and E represent the depth axis coordinates of feature points A, B, C, D, and E on the i-th V-section of the weld, respectively; n is the number of feature points, and h is the preset weld thickness. The optimized weld bead thickness is given by w, which is the preset weld bead width. These represent the transverse axis coordinates of feature points A, B, D, and E on the i-th V-section of the weld, respectively. The coordinate calculation module is used to optimize the number of welding layers, number of weld beads, weld bead thickness, and weld bead width, and then calculate the coordinates of each welding path.

[0052] In one embodiment, the welding control module includes a matrix construction module and a pose generation module, wherein: The matrix construction module is used to assign weights to the neighborhood point clouds of each point on the welding path coordinates to obtain weighted neighborhood point cloud data. After decentralizing the weighted neighborhood point cloud data, a weighted covariance matrix is ​​constructed. The pose generation module is used to perform singular value decomposition on the weighted covariance matrix, take the eigenvector corresponding to the smallest eigenvalue as the normal vector of the target point, and use the normal vector of the target point as the pose data; the target point is any point on the welding path coordinate.

[0053] The foregoing has described one embodiment of the present invention in detail, but this content is merely a preferred embodiment and should not be considered as limiting the scope of the present invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the scope of the claims of this invention.

Claims

1. A control method for a steel pipe welding robot based on machine vision, characterized in that, The method includes: Point cloud data of the weld seam area between the steel pipe and the object to be welded is collected, and the point cloud data is segmented and output as point cloud data of each region; the region includes planar region and curved surface region; The target analysis domain of the weld point cloud is identified by the preset algorithm for the point cloud data of each region, and the weld feature point set is output by filtering the distance feature points. The interpolation curve of the weld feature point set is constructed according to the interpolation function. The welding path coordinates are calculated based on the interpolation curve, and the position data of the welding gun end at each point on the welding path coordinates are obtained by calculating the welding path coordinates. Welding instructions are generated using the welding path and the pose data, and the robot is controlled to perform welding operations on the object to be welded according to the welding instructions.

2. The control method for a steel pipe welding robot based on machine vision according to claim 1, characterized in that, The point cloud data is segmented and processed to output point cloud data for each region, including: Pre-set the maximum number of iterations, the distance threshold from a point to the plane, the threshold for the number of interior points, the threshold for the cluster radius, and the threshold for the number of neighborhood points; In each iteration, three non-collinear points are randomly selected to fit the plane equation as the first plane region. The distance from all points to the first plane region is calculated. All distances are traversed, and points with a distance less than the distance threshold are determined as interior points. If the number of interior points is greater than the number threshold, the first plane region is retained and all interior points are extracted as a plane region. Repeat the iteration until all regions that can be fitted to a plane are segmented, and remove the segmented interior points from the original point cloud data to obtain the residual point cloud that has not been fitted to a plane. Traverse each point p in the residual point cloud, count the number of points in the neighborhood centered at point p with a clustering radius threshold as the radius. If the number of points in the neighborhood is greater than the neighborhood point count threshold, then all points in the neighborhood are grouped into the same cluster, and the operation is repeated until all residual points are clustered and the segmented point cloud data of each region is output.

3. The machine vision-based control method for steel pipe welding robots according to claim 1, characterized in that, Based on the interpolation function, the interpolation curves of the weld feature points are constructed and output, including: The point cloud data of each region is filtered by height and normal vector. If the height difference between the point cloud and the substrate plane in the weld seam area of ​​the medium and thick plate is lower than the height difference threshold, and the angle between the normal vector and the substrate plane is within the angle threshold, then the weld seam candidate area is selected and expanded into the target analysis domain of the point cloud. Two adjacent planar regions within the target analysis domain are selected and defined as the second planar region and the third planar region, respectively. The distance from each point in the third planar region to the second planar region is calculated. Points with a distance less than the distance threshold are taken as weld feature points. Weld feature points that meet the conditions are collected to obtain the weld feature point set. The interpolation curve of the weld feature points is obtained by fitting each feature point in the weld feature point set.

4. The machine vision-based control method for steel pipe welding robots according to claim 1, characterized in that, The welding path coordinates are calculated based on the interpolation curve, including: Obtain the coordinates of the feature points of the weld in the interpolation curve, the preset weld thickness, and the weld width; the feature points of the weld include the two upper endpoints A and B of the V-shaped weld, the bottom endpoint C of the V-shaped weld, and the point D, the midpoint between endpoints A and C, and the point E, the midpoint between endpoints A and B; Substituting these values ​​into the formula, the welding path coordinates of the weld are calculated as follows: in, For the number of welding layers, The width of the m-th layer cross section. This refers to the number of weld passes. Let A, B, C, D, and E represent the depth axis coordinates of feature points A, B, C, D, and E on the i-th V-section of the weld, respectively; n is the number of feature points, and h is the preset weld thickness. The optimized weld bead thickness is given by w, which is the preset weld bead width. These represent the transverse axis coordinates of feature points A, B, D, and E on the i-th V-section of the weld, respectively. After optimizing the number of welding layers, number of weld passes, weld pass thickness, and weld pass width, the coordinates of each weld pass path are calculated.

5. The control method for a steel pipe welding robot based on machine vision according to claim 1, characterized in that, The pose data of the welding torch tip at each point on the welding path coordinates is obtained by calculation, including: Weighted neighborhood point cloud data is obtained by assigning weights to the neighborhood point cloud of each point on the welding path coordinates. After decentralizing the weighted neighborhood point cloud data, a weighted covariance matrix is ​​constructed. Singular value decomposition is performed on the weighted covariance matrix, and the eigenvector corresponding to the smallest eigenvalue is taken as the normal vector of the target point. The normal vector of the target point is then used as the pose data. The target point is any point on the welding path coordinates.

6. A machine vision-based control system for a steel pipe welding robot, characterized in that, The system includes a segmentation module, a construction module, a path calculation module, and a welding control module, wherein: The segmentation module is used to collect point cloud data of the weld seam area between the steel pipe and the object to be welded, and to segment the point cloud data to output point cloud data of each region; the region includes planar region and curved surface region. The construction module is used to identify the target analysis domain of the weld point cloud from the point cloud data of each region through a preset algorithm, and to filter the distance feature points to output the weld feature point set. The interpolation curve of the weld feature point set is constructed and output according to the interpolation function. The path calculation module is used to calculate the welding path coordinates based on the interpolation curve, and to obtain the pose data of the welding gun end at each point on the welding path coordinates through the welding path coordinates. The welding control module is used to generate welding instructions based on the welding path and the pose data, and to control the robot to perform welding operations on the object to be welded according to the welding instructions.

7. The machine vision-based steel pipe welding robot control system according to claim 6, characterized in that, The segmentation module includes an interior point determination module, a removal module, and a region division module, wherein: Pre-set the maximum number of iterations, the distance threshold from a point to the plane, the threshold for the number of interior points, the threshold for the cluster radius, and the threshold for the number of neighborhood points; The interior point determination module is used to randomly select 3 non-collinear points in each iteration to fit the plane equation as the first plane region, calculate the distance from all points to the first plane region, traverse all distances, and determine the points whose distance is less than the distance threshold as interior points. If the number of interior points is greater than the number threshold, the first plane region is retained and all interior points are extracted as a plane region. The removal module is used to iterate repeatedly until all regions that can be fitted to a plane are segmented, and at the same time remove the segmented interior points from the original point cloud data to obtain the residual point cloud that has not been fitted to a plane. The region segmentation module is used to traverse each point p in the residual point cloud, count the number of points in the neighborhood centered on point p and with a clustering radius threshold as the radius. If the number of points in the neighborhood is greater than the neighborhood point count threshold, then all points in the neighborhood are grouped into the same cluster, and the operation is repeated until all residual points are clustered and the segmented point cloud data of each region is output.

8. A machine vision-based steel pipe welding robot control system according to claim 6, characterized in that, The construction module includes a filtering module, a calculation module, and a fitting module, wherein: The filtering module is used to filter the point cloud data of each region by height and normal vector. If the height difference between the point cloud of the weld seam region of the medium and thick plate and the substrate plane is lower than the height difference threshold, and the angle between the normal vector and the substrate plane is within the angle threshold, then the weld seam candidate region is filtered out and expanded into the target analysis domain of the point cloud. The calculation module is used to select two adjacent planar regions within the target analysis domain and define them as the second planar region and the third planar region, respectively. It calculates the distance from each point in the third planar region to the second planar region, takes points whose distance is less than the distance threshold as weld feature points, and collects weld feature points that meet the conditions to obtain a weld feature point set. The fitting module is used to fit each feature point in the weld feature point set to obtain the weld feature point interpolation curve.

9. A machine vision-based steel pipe welding robot control system according to claim 6, characterized in that, The path calculation module includes a parameter calculation module and a coordinate calculation module, wherein: Obtain the coordinates of the feature points of the weld in the interpolation curve, the preset weld thickness, and the weld width; the feature points of the weld include the two upper endpoints A and B of the V-shaped weld, the bottom endpoint C of the V-shaped weld, and the point D, the midpoint between endpoints A and C, and the point E, the midpoint between endpoints A and B; The parameter calculation module is used to substitute these parameters into a formula to calculate the welding path coordinates of the weld. The process is as follows: in, For the number of welding layers, The width of the m-th layer cross section. This refers to the number of weld passes. Let A, B, C, D, and E represent the depth axis coordinates of feature points A, B, C, D, and E on the i-th V-section of the weld, respectively; n is the number of feature points, and h is the preset weld thickness. The optimized weld bead thickness is given by w, which is the preset weld bead width. These represent the transverse axis coordinates of feature points A, B, D, and E on the i-th V-section of the weld, respectively. The coordinate calculation module is used to optimize the number of welding layers, the number of weld passes, the weld pass thickness, and the weld pass width, and then calculate the coordinates of each welding path.

10. A machine vision-based steel pipe welding robot control system according to claim 6, characterized in that, The welding control module includes a matrix construction module and a pose generation module, wherein: The matrix construction module is used to assign weights to the neighborhood point cloud of each point on the welding path coordinate to obtain weighted neighborhood point cloud data, and to construct a weighted covariance matrix after decentralizing the weighted neighborhood point cloud data. The pose generation module is used to perform singular value decomposition on the weighted covariance matrix, take the eigenvector corresponding to the smallest eigenvalue as the normal vector of the target point, and use the normal vector of the target point as pose data; the target point is any point on the welding path coordinates.

Images