An automatic recognition method and system for welding points of steel bar meshes based on point clouds

Abstract

The present invention relates to a method and system for automatically identifying welding points of a steel bar mesh based on point cloud, belonging to the technical field of automatic welding. The method of the present invention is based on three-dimensional point cloud data containing depth information, and uses methods such as clustering and cylinder fitting to achieve the identification of welding points, overcoming the defects of high complexity and low accuracy of the off-line programming method, and overcoming the defect of low accuracy of the method equipped with a 2D camera, and provides an automatic welding point identification method with simple implementation and high positioning accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic welding, and particularly to a method and system for automatically identifying welding points of a steel bar mesh based on point cloud. Background Art

[0002] With the development of the social and economic level, the country has widely carried out the construction of large-scale infrastructure such as bridges and dams, and structural components such as steel bar meshes are widely used therein. The traditional production method of steel bar meshes is manually welded by workers. That is, first, the steel bars are fixed into a mesh structure by a tooling, and then the cross-lap joints of the steel bars are welded by manually operating a welding torch. In the actual operation process, affected by the clamping error of the fixture and human factors, the welding points are prone to problems such as unreliable quality and poor consistency, resulting in uneven shapes and qualities of the welded steel bar meshes, affecting the quality of subsequent engineering construction. In addition, the cost of manual operation is continuously increasing, and the efficiency of manual construction is increasingly difficult to meet the needs of large-scale infrastructure construction. The existing solutions include the following two:

[0003] Solution 1: A welding torch is clamped at the end of a robot, and the welding path of the robot is off-line planned by an off-line programming method according to the design model of the steel bar mesh or by a manual teaching method, so as to realize the automatic welding of a specific type of steel bar mesh. When welding repeatedly, the structure of the steel bar mesh is generally ensured to be unchanged by a tooling fixture, so that the off-line programming path can be reused.

[0004] Solution 2: A 2D camera and a welding torch are clamped at the end of a robot. The camera takes images, and methods such as deep learning are used to identify the target points (steel bar intersections) in the images, and then the welding torch at the end of the robot is guided to move to the welding points to complete the welding action.

[0005] For Solution 1, generally, off-line programming based on the model of the steel bar mesh is adopted, or the welding points of the clamped steel bar mesh are taught and programmed by a manual teaching method, and the welding program of the same type of steel bar mesh is ensured to be reusable by the positioning accuracy of the tooling. The main disadvantages of this method are mainly that a large amount of time needs to be spent in advance to program and plan each type of steel bar mesh, and there are many types of steel bar meshes used in the same project, resulting in too long preparation time; in addition, due to the low dimensional accuracy of the steel bar mesh (generally ±5 mm), the fixed robot welding path is prone to welding point misalignment when welding the same type of steel bar mesh, resulting in unreliable welding quality.

[0006] For Solution 2, it is a new method to mount a vision unit such as a camera at the end of the robot to collect local images of the steel bar mesh, automatically identify the steel bar welding features in the images through mature methods such as deep learning, and guide the robot to automatically perform welding. This method takes a two-dimensional image as the input. Since the image lacks depth information, the robot needs to capture the steel bar mesh at a fixed distance during welding and use this distance as a known quantity to calculate the depth information of the welding features in the image. The main drawback of this method is that the image lacks depth information, and the depth information calculated from the fixed shooting distance is greatly affected by the accuracy of this distance. When the shooting distance changes, it will be reflected proportionally in the depths of all feature points, resulting in difficulty in ensuring the positioning accuracy of robot welding. Summary of the Invention

[0007] In view of this, the present invention provides a method and system for automatically identifying welding points of a steel bar mesh based on point cloud to provide a method for automatically identifying welding points with simple implementation and high positioning accuracy.

[0008] To achieve the above object, the present invention provides the following solutions:

[0009] A method for automatically identifying welding points of a steel bar mesh based on point cloud, the method comprising the following steps:

[0010] Obtain a three-dimensional point cloud data set of a local area of the steel bar mesh;

[0011] Cluster the three-dimensional point cloud data in the three-dimensional point cloud data set, and delete the isolated three-dimensional point cloud data to obtain a plurality of first point cloud clusters;

[0012] Perform a cylindrical fitting on each first point cloud cluster once, and delete the first point cloud clusters with non-steel bar features to obtain first point cloud clusters with steel bar features and the corresponding cylindrical fitting results for each first point cloud cluster with steel bar features;

[0013] According to the corresponding cylindrical fitting results of each first point cloud cluster with steel bar features, merge the first point cloud clusters with steel bar features belonging to the same steel bar to obtain a second point cloud cluster for each steel bar;

[0014] Perform a secondary cylindrical fitting on the second point cloud cluster of each steel bar to obtain a cylindrical point cloud of each steel bar;

[0015] Match the cylindrical point cloud of each steel bar with the design model of the steel bar respectively to obtain a model point cloud of each steel bar; the model point cloud of the steel bar is used to represent the pose of the design model of the steel bar in the steel bar mesh;

[0016] Determine the contact points of different steel bars as welding points according to the model point cloud of each steel bar.

[0017] Optionally, clustering the 3D point cloud data in the 3D point cloud data set and deleting the isolated 3D point cloud data to obtain multiple first point cloud clusters, specifically including:

[0018] Using the Euclidean clustering method to cluster the 3D point cloud data in the 3D point cloud data set and deleting the isolated 3D point cloud data to obtain multiple first point cloud clusters.

[0019] Optionally, performing a cylindrical fitting on each first point cloud cluster once and deleting the first point cloud clusters with non-rebar features to obtain the first point cloud clusters with rebar features and the corresponding cylindrical fitting results for each first point cloud cluster with rebar features, specifically including:

[0020] Let the value of i be 1;

[0021] Using the principal component analysis method to calculate the maximum eigenvector of the covariance matrix of the i-th first point cloud cluster as the axis unit vector of the fitting cylinder;

[0022] Projecting each 3D point cloud data in the i-th first point cloud cluster onto the vertical plane of the axis unit vector to obtain plane point cloud data;

[0023] Performing least squares circle fitting on the plane point cloud data to obtain the center point and radius of the fitting cylinder;

[0024] Judging whether the absolute value of the difference between the radius and the standard radius of the rebar part is greater than a first preset threshold to obtain a judgment result;

[0025] When the judgment result indicates yes, deleting the first point cloud cluster as the first point cloud cluster with non-rebar features;

[0026] When the judgment result indicates no, retaining the first point cloud cluster as the first point cloud cluster with rebar features;

[0027] Let the value of i increase by 1, and return to the step "Using the principal component analysis method to calculate the maximum eigenvector of the covariance matrix of the i-th first point cloud cluster as the axis unit vector of the fitting cylinder", until the value of i is greater than the total number of all first point cloud clusters, outputting the first point cloud clusters with rebar features and the corresponding cylindrical fitting results for each first point cloud cluster with rebar features; the cylindrical fitting results include the axis unit vector, center point and radius of the fitting cylinder.

[0028] Optionally, according to the corresponding cylindrical fitting results of each first point cloud cluster with rebar features, merging the first point cloud clusters with rebar features belonging to the same rebar to obtain the second point cloud cluster of each rebar, specifically including:

[0029] According to the cylindrical fitting results corresponding to the first point cloud clustering of each steel bar feature, use the following formula to calculate the axis angle between the fitting cylinders of the first point cloud clustering of two steel bar features under each combination result; the combination result is obtained by combining any two of the first point cloud clusterings of the steel bar features located in the lower layer as a group.

[0030] σ = l1·l2;

[0031] Where σ represents the axis angle, and l1 and l2 respectively represent the axis unit vectors of the fitting cylinders of the first point cloud clustering of two steel bar features under different combination results.

[0032] Judge whether the formula σ > 0.9 holds to obtain the first judgment result for each combination result.

[0033] According to the cylindrical fitting results corresponding to the first point cloud clustering of each steel bar feature, use the following formula to calculate the shortest distance between the axes of the fitting cylinders of the first point cloud clustering of two steel bar features under each combination result.

[0034] d = |p1 - p2|sin(arccos((p1 - p2)·l1) / (|p1 - p2||l1|));

[0035] Where d represents the shortest distance, p1 and p2 respectively represent the points on the axes of the fitting cylinders of the first point cloud clustering of two steel bar features under different combination results, and l1 represents the axis unit vector of the fitting cylinder of the first point cloud clustering of one of the steel bar features under different combination results.

[0036] Judge whether the formula d < λ holds to obtain the second judgment result for each combination result; where λ represents the axis distance threshold.

[0037] Merge the first point cloud clusterings of the two steel bar feature points under the combination results where the first judgment result is yes and the second judgment result is yes.

[0038] Optionally, determining the contact points of different steel bars as welding points according to the model point cloud of each steel bar specifically includes:

[0039] According to the model point cloud of each steel bar, construct the space straight line equation of each steel bar.

[0040] Based on the space straight line equations of each steel bar, determine the intersection point pairs between any two steel bars; the intersection point pairs include the two points with the shortest distance respectively located on the space straight line equations of any two steel bars.

[0041] Calculate the distances between each intersection point pair respectively.

[0042] Take the midpoint of two points in the intersection point pairs with a distance less than the second preset threshold as the welding point.

[0043] Optionally, after determining the contact points of different steel bars as welding points according to the model point clouds of each steel bar, it further includes:

[0044] Sort all the welding points in ascending order of the distance between the welding points and the welding tool at the end of the welding robot to generate welding path points.

[0045] An automatic recognition system for welding points of a steel bar mesh based on point clouds, the system includes:

[0046] A three-dimensional point cloud data acquisition module for acquiring a set of three-dimensional point cloud data of a local area of the steel bar mesh;

[0047] A clustering module for clustering the three-dimensional point cloud data in the set of three-dimensional point cloud data, and deleting the isolated three-dimensional point cloud data to obtain multiple first point cloud clusters;

[0048] A primary cylinder fitting module for respectively performing primary cylinder fitting on each first point cloud cluster, and deleting the first point cloud clusters with non-steel bar features to obtain the first point cloud clusters with steel bar features and the corresponding cylinder fitting results of each first point cloud cluster with steel bar features;

[0049] A merging module for merging the first point cloud clusters with steel bar features belonging to the same steel bar according to the corresponding cylinder fitting results of each first point cloud cluster with steel bar features to obtain the second point cloud cluster of each steel bar;

[0050] A secondary cylinder fitting module for performing secondary cylinder fitting on the second point cloud cluster of each steel bar to obtain the cylinder point cloud of each steel bar;

[0051] A model matching module for respectively matching the cylinder point cloud of each steel bar with the design model of the steel bar to obtain the model point cloud of each steel bar; the model point cloud of the steel bar is used to represent the pose of the design model of the steel bar in the steel bar mesh;

[0052] A welding point determination module for determining the contact points of different steel bars as welding points according to the model point clouds of each steel bar.

[0053] Optionally, the clustering module specifically includes:

[0054] A clustering sub-module for clustering the three-dimensional point cloud data in the set of three-dimensional point cloud data by using the Euclidean clustering method, and deleting the isolated three-dimensional point cloud data to obtain multiple first point cloud clusters.

[0055] Optionally, the primary cylinder fitting module specifically includes:

[0056] An initialization sub-module for setting the value of i to 1;

[0057] An axis unit vector determination sub-module for calculating the maximum eigenvector of the covariance matrix of the i-th first point cloud cluster as the axis unit vector of the fitting cylinder using the principal component analysis method;

[0058] A projection sub-module for projecting each three-dimensional point cloud data in the i-th first point cloud cluster onto the vertical plane of the axis unit vector to obtain planar point cloud data;

[0059] A least squares circle fitting sub-module for performing least squares circle fitting on the planar point cloud data to obtain the center point and radius of the fitting cylinder;

[0060] A judgment sub-module for judging whether the absolute value of the difference between the radius and the standard radius of the steel bar part is greater than a first preset threshold to obtain a judgment result;

[0061] A processing sub-module for, when the judgment result indicates yes, deleting the first point cloud cluster as a non-steel bar feature; when the judgment result indicates no, retaining the first point cloud cluster as a first point cloud cluster of steel bar features;

[0062] A return sub-module for increasing the value of i by 1 and returning to the step of "calculating the maximum eigenvector of the covariance matrix of the i-th first point cloud cluster as the axis unit vector of the fitting cylinder" until the value of i is greater than the total number of all first point cloud clusters, outputting the first point cloud clusters of steel bar features and the corresponding cylinder fitting results for each first point cloud cluster of steel bar features; the cylinder fitting results include the axis unit vector, center point, and radius of the fitting cylinder.

[0063] Optionally, the merging module specifically includes:

[0064] An axis angle calculation sub-module for calculating the axis angle between the fitting cylinders of two first point cloud clusters of steel bar features under each combination result according to the cylinder fitting results corresponding to each first point cloud cluster of steel bar features using the following formula; the combination results are obtained by combining any two of the first point cloud clusters of steel bar features located in the lower layer as a group;

[0065] σ = l1·l2;

[0066] where σ represents the axis angle, and l1 and l2 respectively represent the axis unit vectors of the fitting cylinders of two first point cloud clusters of steel bar features under different combination results;

[0067] The first judgment result acquisition sub-module is used to judge whether the formula σ > 0.9 holds and obtain the first judgment result of each combination result;

[0068] The shortest distance calculation sub-module is used to calculate the shortest distance between the axes of the fitted cylinders of the first point cloud clusters of two steel bar features under each combination result according to the cylinder fitting results corresponding to the first point cloud clusters of each steel bar feature, using the following formula;

[0069] d = |p1 - p2|sin(arccos((p1 - p2)·l1) / (|p1 - p2||l1|));

[0070] Where, d represents the shortest distance, p1 and p2 respectively represent the points on the axes of the fitted cylinders of the first point cloud clusters of two steel bar features under different combination results, and l1 represents the unit vector of the axis of the fitted cylinder of the first point cloud cluster of one of the steel bar features under different combination results;

[0071] The second judgment result acquisition sub-module is used to judge whether the formula d < λ holds and obtain the second judgment result of each combination result; where, λ represents the axis distance threshold;

[0072] The merging sub-module is used to merge the first point cloud clusters of the two steel bar feature points under the combination result where the first judgment result indicates yes and the second judgment result indicates yes.

[0073] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0074] The present invention discloses an automatic recognition method and system for welding points of a steel bar mesh based on point cloud. The method includes the following steps: obtaining a three-dimensional point cloud data set of a local area of the steel bar mesh; clustering the three-dimensional point cloud data in the three-dimensional point cloud data set and deleting the isolated three-dimensional point cloud data to obtain a plurality of first point cloud clusters; performing cylindrical fitting on each first point cloud cluster once and deleting the first point cloud clusters with non-steel bar features to obtain first point cloud clusters with steel bar features and the corresponding cylindrical fitting results of each first point cloud cluster with steel bar features; merging the first point cloud clusters with steel bar features belonging to the same steel bar according to the corresponding cylindrical fitting results of each first point cloud cluster with steel bar features to obtain second point cloud clusters of each steel bar; performing secondary cylindrical fitting on the second point cloud cluster of each steel bar to obtain a cylindrical point cloud of each steel bar; respectively matching the cylindrical point cloud of each steel bar with the design model of the steel bar to obtain a model point cloud of each steel bar; the model point cloud of the steel bar is used to represent the pose of the design model of the steel bar in the steel bar mesh; determining the contact points of different steel bars as welding points according to the model point cloud of each steel bar. The present invention is based on three-dimensional point cloud data containing depth information and uses methods such as clustering and cylindrical fitting to realize the recognition of welding points, overcomes the defects of high complexity and low accuracy of the off-line programming method, and overcomes the defect of low accuracy of the method with a 2D camera mounted, and provides an automatic recognition method for welding points with simple implementation and high positioning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to these drawings.

[0076] Figure 1 It is a flowchart of an automatic recognition method for welding points of a steel bar mesh based on point cloud provided in Embodiment 1 of the present invention;

[0077] Figure 2 It is a schematic principle diagram of an automatic recognition method for welding points of a steel bar mesh based on point cloud provided in Embodiment 1 of the present invention;

[0078] Figure 3 It is a schematic structural diagram of obtaining three-dimensional point cloud data provided in Embodiment 1 of the present invention;

[0079] Figure 4 It is a distribution diagram of three-dimensional point cloud data in a three-dimensional point cloud data set provided in Embodiment 1 of the present invention;

[0080] Figure 5It is the three-dimensional point cloud data distribution diagram in the first point cloud clustering provided by Embodiment 1 of the present invention. Detailed implementation manners

[0081] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present invention.

[0082] The purpose of the present invention is to provide a method and system for automatically identifying the welding points of a steel bar mesh based on point cloud, so as to provide an automatic welding point identification method with simple implementation and high positioning accuracy.

[0083] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in conjunction with the drawings and specific implementation manners.

[0084] The present invention provides a method for automatically identifying the welding points of a steel bar mesh based on point cloud. The method includes the following steps: obtaining a set of three-dimensional point cloud data of a local area of the steel bar mesh; clustering the three-dimensional point cloud data in the set of three-dimensional point cloud data, and deleting the isolated three-dimensional point cloud data to obtain a plurality of first point cloud clusters; performing a cylindrical fitting on each first point cloud cluster once to obtain a cylindrical fitting result; wherein, the cylindrical fitting result of any first point cloud cluster is at least used to: determine that the first point cloud cluster conforms to non-steel bar characteristics or conforms to steel bar characteristics; deleting the first point cloud clusters that conform to non-steel bar characteristics to obtain all the first point cloud clusters that conform to steel bar characteristics; the first point cloud clusters that conform to steel bar characteristics are target point cloud clusters; according to the cylindrical fitting results of the target point cloud clusters, merging the target point cloud clusters belonging to the same steel bar to obtain a second point cloud cluster for each steel bar; performing a secondary cylindrical fitting on the second point cloud cluster of each steel bar to obtain a cylindrical point cloud for each steel bar; respectively matching the cylindrical point cloud of each steel bar with the design model of the steel bar to obtain a model point cloud for each steel bar; the model point cloud of the steel bar is used to represent the pose of the steel bar in the steel bar mesh; determining the contact points of different steel bars as welding points according to the model point cloud of each steel bar.

[0085] Embodiment 1

[0086] As Figure 1 and 2 shown, Embodiment 1 of the present invention provides a method for automatically identifying the welding points of a steel bar mesh based on point cloud. This method takes the three-dimensional point cloud data scanned by a 3D camera as input, automatically identifies the intersection characteristics of the steel bar mesh, and outputs the welding points and normal vectors to guide the automatic welding of the robot welding system.

[0087] The method includes the following steps:

[0088] Step 101: Obtain a set of three-dimensional point cloud data for a local area of the steel bar mesh.

[0089] As Figure 3 shown, the welding robot 1 drives the 3D camera 2 to scan a local area of the steel bar mesh 4 to obtain the three-dimensional point cloud data as Figure 4 shown. Figure 3 The welding torch 3 in

[0090] is used as the welding tool of the welding robot.

[0091] Step 102: Cluster the three-dimensional point cloud data in the set of three-dimensional point cloud data, and delete the isolated three-dimensional point cloud data to obtain multiple first point cloud clusters.

[0092] Step 102 is the preliminary segmentation of the three-dimensional point cloud data and the removal of isolated noise points. The collected three-dimensional point cloud data is scattered and disordered, and there are many isolated noise points. To achieve the segmentation of the steel bar point cloud and remove the isolated noise points, Embodiment 1 of the present invention uses Euclidean clustering to segment and denoise the three-dimensional point cloud data, sets the threshold of Euclidean clustering as t, and sets the minimum number of point clouds in a single cluster as N, so as to remove all clusters with the number of points less than N (i.e., isolated noise points) while segmenting the point cloud, realizing the preliminary segmentation of the three-dimensional point cloud data and the removal of isolated noise points; finally, this step outputs multiple first point cloud clusters after removing the isolated noise points.

[0092] Step 103: Perform a cylindrical fitting on each first point cloud cluster respectively, and delete the first point cloud clusters with non-steel bar features to obtain the first point cloud clusters with steel bar features and the corresponding cylindrical fitting results for each first point cloud cluster with steel bar features.

[0093] Step 103 includes point cloud cylindrical fitting based on the least squares method and deleting clusters with non-steel bar features.

[0094] Point cloud cylindrical fitting based on the least squares method. Taking the result of Step 102 as the input, considering that the steel bar is generally an approximate cylindrical feature with a large aspect ratio, the three-dimensional point cloud data of a single steel bar in a single cluster is generally distributed in a long and narrow shape. The principal component analysis method is used to calculate the maximum eigenvector of the covariance matrix of the three-dimensional point cloud data in this first point cloud cluster, which is the axis unit vector l = [l, m, n] of the fitted cylinder; then project the three-dimensional point cloud data in this first point cloud cluster onto a plane perpendicular to the axis, perform least squares circle fitting on the projected plane point cloud data, and calculate the center point p = [x, y, z] (which is a point on the axis) and the radius R; draw a cylindrical point cloud based on the center point p, the axis vector l, and the radius R, and output the cylindrical point cloud fitted by each first point cloud cluster, as well as the corresponding point p on the axis, the axis vector l, and the radius R.

[0095] Delete the clustering of non-rebar features. Using the cylindrical point cloud and radius R obtained by fitting each first point cloud clustering as input, set the standard radius of the rebar part as R0 and set the threshold δ. If |R - R0| > δ, it is considered that the clustering point cloud belongs to noise points, and delete this first point cloud clustering; finally, output the first point cloud clustering of the rebar features.

[0096] Step 104: According to the cylindrical fitting results corresponding to the first point cloud clustering of each rebar feature, merge the first point cloud clustering of the rebar features belonging to the same rebar to obtain the second point cloud clustering of each rebar.

[0097] Step 104 is used to merge multiple first point cloud clusterings of the same rebar. This step takes the first point cloud clustering of the rebar features output by step 103 and a point p on the corresponding axis, axis vector l, and radius R as input; when any two rebars cross, the upper rebar will block the lower rebar, resulting in over-segmentation of the three-dimensional point cloud data of the lower rebar (as Figure 5 shown), so it is necessary to merge multiple first point cloud clusterings of the same rebar in the lower layer. For a total of n groups of point cloud clusterings, there are

[0098] combinations when arbitrarily selecting two of the first point cloud clusterings. Next, it is necessary to determine whether the axes of the fitted cylinders corresponding to any two first point cloud clusterings are approximately coincident. If they are approximately coincident, merge the corresponding multiple point cloud clusterings to achieve one second point cloud clustering for each rebar.

[0099] 1) Taking the first point cloud clustering of the rebar features as input, generate combinations of pairwise point cloud clusterings;

[0100] 2) Select one combination from the

[0101] combinations. Let a point on the axis of the fitted cylinder corresponding to two of the first point cloud clusterings be p1 and p2 respectively, the axis vectors be l1 and l2 respectively, and the radii be R1 and R2;

[0102] 3) Calculate the axis angle σ = l1·l2. If σ > 0.9, go to 4); otherwise, go to 2);

[0103] d = |p1 - p2|sin(arccos((p1 - p2)·l1) / (|p1 - p2||l1|))

[0104] 5) If d < λ (where λ represents the threshold of the axis distance), it is considered that axis 2 and axis 1 approximately coincide, and the first point cloud clustering of the corresponding steel bar features is merged;

[0105] Determine whether all combinations have been traversed. If not, go to 2). If all have been traversed, the algorithm ends and the merged second point cloud clustering is output.

[0106] Step 105, perform secondary cylindrical fitting on the second point cloud clustering of each steel bar to obtain the cylindrical point cloud of each steel bar.

[0107] Perform cylindrical fitting on the merged second point cloud clustering again. This step takes the merged second point cloud clustering output by step 104 as the input, and uses the point cloud cylindrical fitting method based on the least squares method in step 103 to perform cylindrical fitting again. Finally, the cylindrical point cloud obtained by fitting each second point cloud clustering, as well as a point p' on the corresponding axis, axis vector l', and radius R' are output;

[0108] Step 106, respectively match the cylindrical point cloud of each steel bar with the design model of the steel bar to obtain the model point cloud of each steel bar; the model point cloud of the steel bar is used to characterize the pose of the design model of the steel bar in the steel bar mesh.

[0109] This step takes the cylindrical point cloud output by step 105 and the design model of a single steel bar as the input, and uses the point cloud - model matching algorithm of ICP (Iterative Closest Point) to perform three - dimensional matching between the design model of the steel bar and the cylindrical point cloud of the steel bar respectively. Calculate the pose transformation matrix T from the coordinate system of the design model of the steel bar to the cylindrical point cloud, so as to transform the design model of the steel bar to fit the cylindrical point cloud of the steel bar, and then readjust the length of the design model of the steel bar to make it consistent with the length of the corresponding cylindrical point cloud, as Figure 5 shown; this step finally outputs the model point cloud of all the steel bars after matching, as well as the corresponding unit vector l″ of the steel bar axis and a point p″ on the axis.

[0110] Step 107, determine the contact points of different steel bars as welding points according to the model point cloud of each steel bar.

[0111] This step takes the model point cloud of the steel bar output by step 106, as well as the corresponding unit vector l″ of the steel bar axis and a point p″ on the axis as the input. For the model point cloud of n groups of steel bars, n space straight - line equations can be constructed according to the corresponding l″ and p″, and there are types of pairwise combinations; calculate the closest - point pairs (intersection - point pairs) between the space straight - line equations of the steel bars. The specific calculation method is as follows:

[0112] 1) According to the unit vector l″1, l″2 and the point p″1, p″2 on the axis, the space lines are constructed as line1 and line2, and the cross product of the unit vectors l″1 and l″2 of the steel bar axis is obtained to obtain the plane plane1 parallel to the two vectors and its normal vector l″3;

[0113] 2) Calculate the cross product of l″3 and l″1 to obtain the plane plane2 perpendicular to l″3 and passing through l″1 and its normal vector l″4, calculate the intersection of plane2 and line2, and obtain the closest point t2 on line2;

[0114] 3) Calculate the cross product of l″3 and l″2 to obtain the plane plane4 perpendicular to l″3 and passing through l″2 and its normal vector l″5, calculate the intersection of plane4 and line1, and obtain the closest point t1 on line1;

[0115] 4) t1 and t2 are the closest points on the two spatial lines. Output t1 and t2.

[0116] The K nearest neighbor algorithm is used to calculate the shortest distance d from the nearest point t1 and t2 to the model point cloud of the steel bar (i.e., the distance between the nearest points t1 and t2). If d>2R, it means that the nearest point pair of the steel bar axis is not within the field of view of the 3D camera this time, and it is an invalid nearest point pair. Calculate all the closest point pairs and remove invalid closest point pairs to obtain all valid closest point pairs within the field of view of the 3D camera; calculate the midpoint of the closest point pair as the contact point of the two steel bars (i.e., the welding point), and calculate the unit vector between the closest point pairs, i.e., the local normal vector of the steel mesh; finally, this step outputs the coordinates and normal vectors of all welding points within the field of view, as shown in Figure 2 As shown;

[0117] Robot welding path generation. The welding point coordinates and normal vectors are used as input to sort the welding points, and the approach, welding / binding, retraction, and transition paths of the robot end welding tool are generated based on the spatial coordinates of the welding points and the normal vectors of the steel mesh. This step finally outputs the complete robot welding path points.

[0118] Robot welding program post-processing. Taking the robot welding path points as input, post-process the path, add welding trigger instructions, and convert them into robot executable motion programs to achieve welding of all points in the local measurement area.

[0119] Example 2

[0120] Embodiment 2 of the present invention provides a point cloud-based automatic identification system for steel mesh welding points, the system comprising:

[0121] A three-dimensional point cloud data acquisition module, which is used to acquire a set of three-dimensional point cloud data of a local area of a steel bar mesh.

[0122] A clustering module, which is used to cluster the three-dimensional point cloud data in the set of three-dimensional point cloud data, and delete the isolated three-dimensional point cloud data, so as to obtain a plurality of first point cloud clusters.

[0123] The clustering module specifically includes: a clustering sub-module, which is used to cluster the three-dimensional point cloud data in the set of three-dimensional point cloud data by using the Euclidean clustering method, and delete the isolated three-dimensional point cloud data, so as to obtain a plurality of first point cloud clusters.

[0124] A primary cylinder fitting module, which is used to perform primary cylinder fitting on each first point cloud cluster respectively, and delete the first point cloud clusters with non-steel bar features, so as to obtain the first point cloud clusters with steel bar features and the corresponding cylinder fitting results of each first point cloud cluster with steel bar features.

[0125] The primary cylinder fitting module specifically includes: an initialization sub-module, which is used to set the value of i to 1; an axis unit vector determination sub-module, which is used to calculate the maximum eigenvector of the covariance matrix of the i-th first point cloud cluster by using the principal component analysis method as the axis unit vector of the fitting cylinder; a projection sub-module, which is used to project each three-dimensional point cloud data in the i-th first point cloud cluster onto the vertical plane of the axis unit vector to obtain plane point cloud data; a least squares circle fitting sub-module, which is used to perform least squares circle fitting on the plane point cloud data to obtain the center point and radius of the fitting cylinder; a judgment sub-module, which is used to judge whether the absolute value of the difference between the radius and the standard radius of the steel bar part is greater than a first preset threshold to obtain a judgment result; a processing sub-module, which is used to, when the judgment result indicates yes, delete the first point cloud cluster as the first point cloud cluster with non-steel bar features; when the judgment result indicates no, retain the first point cloud cluster as the first point cloud cluster with steel bar features; a return sub-module, which is used to increase the value of i by 1, and return to the step of "calculating the maximum eigenvector of the covariance matrix of the i-th first point cloud cluster by using the principal component analysis method as the axis unit vector of the fitting cylinder", until the value of i is greater than the total number of all first point cloud clusters, and output the first point cloud clusters with steel bar features and the corresponding cylinder fitting results of each first point cloud cluster with steel bar features; the cylinder fitting result includes the axis unit vector, the center point and the radius of the fitting cylinder.

[0126] A merging module, which is used to merge the first point cloud clusters with steel bar features belonging to the same steel bar according to the corresponding cylinder fitting results of each first point cloud cluster with steel bar features, so as to obtain the second point cloud cluster of each steel bar.

[0127] The merging module specifically includes:

[0128] An axis angle calculation sub-module, which is used to calculate the axis angle of the fitting cylinders of the first point cloud clusters of two steel bar features under each combination result according to the cylinder fitting results corresponding to the first point cloud clusters of each steel bar feature, using the following formula; the combination results are obtained by combining any two of the first point cloud clusters of the steel bar features located in the lower layer as a group.

[0129] σ = l1·l2;

[0130] Where σ represents the axis angle, and l1 and l2 respectively represent the axis unit vectors of the fitting cylinders of the first point cloud clusters of two steel bar features under different combination results.

[0131] A first judgment result acquisition sub-module, which is used to judge whether the formula σ > 0.9 holds, and obtain the first judgment result of each combination result.

[0132] A shortest distance calculation sub-module, which is used to calculate the shortest distance between the axes of the fitting cylinders of the first point cloud clusters of two steel bar features under each combination result according to the cylinder fitting results corresponding to the first point cloud clusters of each steel bar feature, using the following formula.

[0133] d = |p1 - p2|sin(arccos((p1 - p2)·l1) / (|p1 - p2||l1|));

[0134] Where d represents the shortest distance, p1 and p2 respectively represent the points on the axes of the fitting cylinders of the first point cloud clusters of two steel bar features under different combination results, and l1 represents the axis unit vector of the fitting cylinder of the first point cloud cluster of one of the steel bar features under different combination results.

[0135] A second judgment result acquisition sub-module, which is used to judge whether the formula d < λ holds, and obtain the second judgment result of each combination result; where λ represents the axis distance threshold.

[0136] A merging sub-module, which is used to merge the first point cloud clusters of the two steel bar feature points under the combination results where the first judgment result is yes and the second judgment result is yes.

[0137] A quadratic cylinder fitting module, which is used to perform quadratic cylinder fitting on the second point cloud clusters of each steel bar to obtain the cylinder point cloud of each steel bar.

[0138] A model matching module, which is used to respectively match the cylinder point cloud of each steel bar with the design model of the steel bar to obtain the model point cloud of each steel bar; the model point cloud of the steel bar is used to characterize the pose of the design model of the steel bar in the steel bar mesh.

[0139] The welding point determination module is used to determine the contact points of different steel bars as welding points according to the model point cloud of each steel bar.

[0140] Based on the above embodiments, the advantages of the present invention are as follows:

[0141] 1. Steel bar point cloud segmentation and isolated noise point deletion. The initial scanned point cloud is a whole, and there are a large number of isolated noise points. How to achieve the point cloud segmentation and denoising corresponding to a single steel bar is a difficult problem. The present invention first uses Euclidean clustering to perform preliminary segmentation and isolated noise point deletion on the point cloud, and then uses point cloud cylinder fitting to further remove the point cloud clusters with non-steel bar features. At the same time, multiple point cloud clusters with approximately coincident fitted cylinder axes are merged to ensure that each steel bar feature corresponds to only one point cloud cluster while removing non-steel bar isolated noise points.

[0142] 2. High-robust matching and positioning of steel bar models based on incomplete steel bar point clouds. The point cloud obtained by a single scan of a 3D camera is only part of the features of the steel bar facing the 3D camera directly. Direct matching with the design model of the steel bar will cause distortion. The present invention first performs spatial cylinder fitting on the segmented incomplete steel bar point cloud to draw a complete cylinder point cloud, and then aligns the steel bar model with the cylinder point cloud through matching methods such as ICP to achieve precise positioning of the steel bar model with high robustness, solving the problem of matching and positioning distortion caused by incomplete point clouds.

[0143] 3. Identification and extraction of the intersection points of steel bar models. Calculate the nearest point pairs (intersection point pairs) between the axes of steel bars pairwise, and use the K-nearest neighbor algorithm to calculate the shortest distance d from the intersection point pairs to the model point cloud of the steel bars. Calculate the effective intersection point pairs within the field of view according to the size of d, and then output the three-dimensional precise coordinates and normal vectors of all welding points within the field of view, overcoming the problem of poor positioning accuracy of welding points based on traditional image recognition.

[0144] In this specification, each embodiment is described in a progressive manner. The key points of each embodiment are the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0145] Specific examples are used in this article to elaborate on the principles and implementation methods of the present invention. The descriptions of the above embodiments are only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. An automatic recognition method for the welding points of a steel bar mesh based on point cloud, characterized in that, The method includes the following steps: Obtain a set of 3D point cloud data for a local area of the steel bar mesh; Cluster the 3D point cloud data in the set of 3D point cloud data, and delete the isolated 3D point cloud data to obtain multiple first point cloud clusters; Perform a cylindrical fitting on each first point cloud cluster respectively, and delete the first point cloud clusters with non-steel bar features to obtain the first point cloud clusters with steel bar features and the corresponding cylindrical fitting results for each first point cloud cluster with steel bar features; According to the corresponding cylindrical fitting results of each first point cloud cluster with steel bar features, merge the first point cloud clusters with steel bar features belonging to the same steel bar to obtain the second point cloud cluster for each steel bar; Perform a secondary cylindrical fitting on the second point cloud cluster for each steel bar to obtain the cylindrical point cloud for each steel bar; Match the cylindrical point cloud for each steel bar with the design model of the steel bar respectively to obtain the model point cloud for each steel bar; The model point cloud of the steel bar is used to represent the pose of the design model of the steel bar in the steel bar mesh; Determine the contact points of different steel bars as welding points according to the model point cloud of each steel bar.

2. The automatic recognition method for the welded joints of the steel bar mesh of the point cloud according to claim 1, characterized in that, The step of clustering the 3D point cloud data in the set of 3D point cloud data and deleting the isolated 3D point cloud data to obtain multiple first point cloud clusters specifically includes: Use the Euclidean clustering method to cluster the 3D point cloud data in the set of 3D point cloud data, and delete the isolated 3D point cloud data to obtain multiple first point cloud clusters.

3. The automatic recognition method for the welded joints of the steel bar mesh of the point cloud according to claim 1, characterized in that, The step of performing a cylindrical fitting on each first point cloud cluster respectively and deleting the first point cloud clusters with non-steel bar features to obtain the first point cloud clusters with steel bar features and the corresponding cylindrical fitting results for each first point cloud cluster with steel bar features specifically includes: Let the value of i be 1; Use the principal component analysis method to calculate the maximum eigenvector of the covariance matrix of the i-th first point cloud cluster as the axis unit vector of the fitting cylinder; Project each 3D point cloud data in the i-th first point cloud cluster onto the vertical plane of the axis unit vector to obtain plane point cloud data; Perform a least squares circle fitting on the plane point cloud data to obtain the center point and radius of the fitting cylinder; Judge whether the absolute value of the difference between the radius and the standard radius of the steel bar part is greater than a first preset threshold to obtain a judgment result; When the judgment result indicates yes, delete the first point cloud cluster as the first point cloud cluster with non-steel bar features; When the judgment result indicates no, retain the first point cloud cluster as the first point cloud cluster with steel bar features; Let the value of i increase by 1, and return to the step "Use the principal component analysis method to calculate the maximum eigenvector of the covariance matrix of the i-th first point cloud cluster as the axis unit vector of the fitting cylinder" until the value of i is greater than the total number of all first point cloud clusters, and output the first point cloud clusters with steel bar features and the corresponding cylindrical fitting results for each first point cloud cluster with steel bar features; the cylindrical fitting results include the axis unit vector, center point and radius of the fitting cylinder.

4. The automatic recognition method for the welded joints of the steel bar mesh of the point cloud according to claim 1, characterized in that, For the cylinder fitting results corresponding to the first point cloud clustering of each steel bar feature, the first point cloud clustering of the steel bar features belonging to the same steel bar are merged to obtain the second point cloud clustering of each steel bar, which specifically includes: Based on the cylinder fitting results corresponding to the first point cloud clustering of each steel bar feature, use the following formula to calculate the axis angle between the fitting cylinders of the first point cloud clustering of two steel bar features under each combination result; the combination results are obtained by combining any two of the first point cloud clustering of the steel bar features located in the lower layer as a group; σ = l1·l2; Where σ represents the axis angle, and l1 and l2 respectively represent the axis unit vectors of the fitting cylinders of the first point cloud clustering of two steel bar features under different combination results; Judge whether the formula σ > 0.9 holds to obtain the first judgment result of each combination result; Based on the cylinder fitting results corresponding to the first point cloud clustering of each steel bar feature, use the following formula to calculate the shortest distance between the axes of the fitting cylinders of the first point cloud clustering of two steel bar features under each combination result; d = |p1 - p2|sin(arccos((p1 - p2)·l1) / (|p1 - p2||l1|)); Where d represents the shortest distance, p1 and p2 respectively represent the points on the axes of the fitting cylinders of the first point cloud clustering of two steel bar features under different combination results, and l1 represents the axis unit vector of the fitting cylinder of the first point cloud clustering of one of the steel bar features under different combination results; Judge whether the formula d < λ holds to obtain the second judgment result of each combination result; where λ represents the axis distance threshold; Merge the first point cloud clustering of the two steel bar feature points under the combination result where the first judgment result is yes and the second judgment result is yes.

5. The automatic recognition method for welding points of steel bar mesh of point cloud according to claim 1, characterized in that, The method for determining the contact points between different steel bars as welding points according to the model point cloud of each steel bar specifically includes: Based on the model point cloud of each steel bar, construct the space straight line equation of each steel bar; Based on the space straight line equations of each steel bar, determine the intersection point pairs between any two steel bars; the intersection point pairs include the two points with the shortest distance respectively located on the space straight line equations of any two steel bars; Calculate the distances between each intersection point pair respectively; Take the midpoint of the two points in the intersection point pair with a distance less than the second preset threshold as the welding point.

6. The automatic recognition method for the welded joints of the steel bar mesh of the point cloud according to claim 1, characterized in that, After determining the contact points between different steel bars as welding points according to the model point cloud of each steel bar, it further includes: Sort all the welding points in ascending order of the distance between the welding points and the welding tool at the end of the welding robot to generate welding path points.

7. An automatic recognition system for the welding points of a steel bar mesh based on point cloud, characterized in that, The system includes: A three-dimensional point cloud data acquisition module for acquiring a three-dimensional point cloud data set of a local area of a steel bar mesh; A clustering module for clustering the three-dimensional point cloud data in the three-dimensional point cloud data set and deleting the isolated three-dimensional point cloud data to obtain a plurality of first point cloud clusterings; A primary cylinder fitting module, which is used to perform primary cylinder fitting on each first point cloud cluster respectively, and delete the first point cloud clusters of non-rebar features, so as to obtain the first point cloud clusters of rebar features and the corresponding cylinder fitting results for each first point cloud cluster of rebar features; A merging module, which is used to merge the first point cloud clusters of rebar features belonging to the same rebar according to the corresponding cylinder fitting results of each first point cloud cluster of rebar features, so as to obtain the second point cloud cluster of each rebar; A secondary cylinder fitting module, which is used to perform secondary cylinder fitting on the second point cloud cluster of each rebar to obtain the cylinder point cloud of each rebar; A model matching module, which is used to match the cylinder point cloud of each rebar with the design model of the rebar respectively to obtain the model point cloud of each rebar; the model point cloud of the rebar is used to represent the pose of the design model of the rebar in the rebar mesh; A welding point determination module, which is used to determine the contact points of different rebars as welding points according to the model point cloud of each rebar.

8. The automatic recognition system for the welded joints of the steel bar mesh of the point cloud according to claim 7, characterized in that, The clustering module specifically includes: A clustering sub-module, which is used to cluster the 3D point cloud data in the 3D point cloud data set by using the Euclidean clustering method and delete the isolated 3D point cloud data to obtain a plurality of first point cloud clusters.

9. The automatic recognition system for the welded joints of the steel bar mesh of the point cloud according to claim 7, characterized in that The primary cylinder fitting module specifically includes: An initialization sub-module, which is used to set the value of i to 1; An axis unit vector determination sub-module, which is used to calculate the maximum eigenvector of the covariance matrix of the i-th first point cloud cluster by using the principal component analysis method as the axis unit vector of the fitting cylinder; A projection sub-module, which is used to project each 3D point cloud data in the i-th first point cloud cluster onto the vertical plane of the axis unit vector to obtain plane point cloud data; A least squares circle fitting sub-module, which is used to perform least squares circle fitting on the plane point cloud data to obtain the center point and radius of the fitting cylinder; A judgment sub-module, which is used to judge whether the absolute value of the difference between the radius and the standard radius of the rebar part is greater than a first preset threshold to obtain a judgment result; A processing sub-module, which is used to delete the first point cloud cluster as the first point cloud cluster of non-rebar features when the judgment result is yes; and retain the first point cloud cluster as the first point cloud cluster of rebar features when the judgment result is no; A return sub-module, which is used to increase the value of i by 1 and return to the step of "calculating the maximum eigenvector of the covariance matrix of the i-th first point cloud cluster by using the principal component analysis method as the axis unit vector of the fitting cylinder" until the value of i is greater than the total number of all first point cloud clusters, and output the first point cloud clusters of rebar features and the corresponding cylinder fitting results for each first point cloud cluster of rebar features; the cylinder fitting results include the axis unit vector, center point and radius of the fitting cylinder.

10. The automatic recognition system for the welded joints of the steel bar mesh of the point cloud according to claim 7, characterized in that, The merging module specifically includes: An axis angle calculation sub-module, which is used to calculate the axis angle of the fitting cylinders of the first point cloud clusters of two steel bar features under each combination result according to the cylinder fitting results corresponding to the first point cloud clusters of each steel bar feature, using the following formula; the combination results are the combination results obtained by combining any two of the first point cloud clusters of the steel bar features located in the lower layer as a group; σ = l1·l2; where σ represents the axis angle, and l1 and l2 respectively represent the axis unit vectors of the fitting cylinders of the first point cloud clusters of two steel bar features under different combination results; A first judgment result acquisition sub-module, which is used to judge whether the formula σ > 0.9 holds, and obtain the first judgment result of each combination result; A shortest distance calculation sub-module, which is used to calculate the shortest distance between the axes of the fitting cylinders of the first point cloud clusters of two steel bar features under each combination result according to the cylinder fitting results corresponding to the first point cloud clusters of each steel bar feature, using the following formula; d = |p1 - p2|sin(arccos((p1 - p2)·l1) / (|p1 - p2||l1|)); where d represents the shortest distance, p1 and p2 respectively represent the points on the axes of the fitting cylinders of the first point cloud clusters of two steel bar features under different combination results, and l1 represents the axis unit vector of the fitting cylinder of the first point cloud cluster of one of the steel bar features under different combination results; A second judgment result acquisition sub-module, which is used to judge whether the formula d < λ holds, and obtain the second judgment result of each combination result; where λ represents the axis distance threshold; A merging sub-module, which is used to merge the first point cloud clusters of the two steel bar feature points under the combination result where the first judgment result is yes and the second judgment result is yes.

Images