Welding track generation method based on three-dimensional point cloud

Abstract

The welding track generation method based on the three-dimensional point cloud specifically comprises the following steps that S1, a model is established, specifically, firstly, a line structure light sensor is used for scanning a welding seam area of a weldment, and ordered point cloud on the surface of a welding seam is obtained; secondly, converting the point cloud data represented in the sensor coordinate system into point cloud data represented in a welding robot base coordinate system to obtain a three-dimensional point cloud model of the welding seam; s2, feature extraction: preprocessing point clouds in the model, firstly removing point clouds outside a set coordinate system range by using a straight-through filter, retaining point clouds within the set coordinate system range, and then removing abnormal values of the point clouds by using a Gaussian filter; secondly, fitting the preprocessed point cloud by adopting an RANSAC (Random Sample Consensus) algorithm, and extracting welding spot data; and S3, attitude estimation is conducted, specifically, based on the welding spot data extracted in the step S2, a welding track is generated through an NURBS curve algorithm, the track generation accuracy is high, and a welding robot can conduct corresponding welding process adjustment according to the actual situation.

Description

Technical Field

[0001] The invention relates to the technical field of robot welding, and in particular to a welding trajectory generation method based on three-dimensional point cloud. Background Art

[0002] In industrial production, the technical level of workers will directly affect the effect of manual welding, which makes it difficult to maintain consistent welding efficiency and quality. In addition, manual welding has a harsh working environment, high repetitiveness, and high labor intensity. In addition, the strong light and harmful substances generated during the welding process can also cause damage to the health of workers. With the continuous breakthroughs in intelligent manufacturing, robot welding has ushered in rapid development. In contrast, intelligent welding has high precision and good quality, which not only significantly improves welding efficiency, but also reduces the labor intensity of workers and reduces the production costs of enterprises. Therefore, it has been widely used in the production of simple, stable, and large-scale workpieces.

[0003] During the welding process, due to the influence of various factors, welds often have a variety of welding defects, such as undercuts, pits, weld nodules and other surface defects; in traditional welding robot systems, weld identification usually relies on manual inspection of the weld area of ​​the workpiece by human eyes. This method is inefficient and highly subjective, and is easily affected by environmental interference, which affects the accuracy of the detection results, leading to misjudgment of the weld defect location, missed detection, and other phenomena, which in turn affects the overall welding quality. Summary of the invention

[0004] The present application provides a welding trajectory generation method based on three-dimensional point cloud, which is used to overcome the above-mentioned problems existing in the prior art. The welding trajectory generation method based on three-dimensional point cloud of the present application first scans the weld area through a line structured light sensor to obtain a three-dimensional point cloud model of the weld, then processes and extracts features from the point cloud in the model, and finally generates a welding trajectory based on the extracted feature data, with high weld recognition accuracy and high trajectory generation accuracy; thus, the welding robot can make corresponding welding process adjustments according to actual conditions, thereby improving welding forming effects and work efficiency.

[0005] The technical solution of this application is:

[0006] A welding trajectory generation method based on three-dimensional point cloud, the method specifically includes the following steps: S1, model establishment: first, use a line structured light sensor to scan the weld area of ​​the weldment to obtain an ordered point cloud on the weld surface; then, convert the point cloud data represented in the sensor coordinate system into point cloud data represented in the welding robot base coordinate system to obtain a three-dimensional point cloud model of the weld; S2, feature extraction: pre-process the point cloud in the model, first use a straight-through filter to remove the point cloud outside the set coordinate system range, retain the point cloud within the set coordinate system range, and then use a Gaussian filter to remove outliers in the point cloud; then, use the RANSAC algorithm to fit the pre-processed point cloud and extract the weld data; S3, posture estimation: based on the weld data extracted in step S2, use a NURBS curve algorithm to generate a welding trajectory.

[0007] Compared with the prior art, the welding trajectory generation method based on three-dimensional point cloud of the present application includes three steps of model establishment, feature extraction and posture estimation. It can effectively identify welds, measure and extract relevant information of welds to generate welding trajectories, and improve welding quality and welding accuracy; specifically: first use a line structured light sensor to scan the weld area on the weldment to effectively identify the weld and obtain a three-dimensional point cloud model of the weld; then use a filtering algorithm to pre-process the point cloud in the model, remove outliers (the initial three-dimensional point cloud model obtained usually has redundancy and noise, which needs to be removed), and extract weld point data; finally, according to the extracted weld point data, a NURBS curve algorithm is used to generate a welding trajectory, and the trajectory generation has high accuracy; thereby, the welding robot can make corresponding welding process adjustments according to actual conditions, thereby improving welding forming effects and work efficiency.

[0008] As an optimization, in the aforementioned method for generating welding trajectories based on three-dimensional point clouds, in step S3, the generated welding trajectories are optimized using the Newton iteration method to obtain the optimal welding trajectories. The Newton iteration method has the advantages of fast convergence speed and high precision, and only a few iterations are required to meet the precision requirements; thus, the optimized welding trajectories are closer to the actual situation, further ensuring the welding quality and welding precision.

[0009] As an optimization, in the aforementioned welding trajectory generation method based on three-dimensional point cloud, in step S1, the data of each scan is regarded as a row of ordered point cloud and numbered, and the scanning order is used as the column number of the ordered point cloud, so as to obtain the ordered point cloud of the weld surface. After the point cloud is arranged in an orderly manner, it is convenient to subsequently establish a three-dimensional point cloud model of the weld.

[0010] Furthermore, the conversion relationship between the sensor coordinate system and the welding robot base coordinate system is: Among them, N BOT Represents the point cloud data in the welding robot base coordinate system {BOT}; NSEN Represents point cloud data in the sensor coordinate system {SEN}; Represents the transformation matrix from the welding robot base coordinate system {BOT} to the intermediate coordinate system {BS}; Represents the transformation matrix from the sensor coordinate system {SEN} to the intermediate coordinate system {BS}.

[0011] As an optimization, in the aforementioned welding trajectory generation method based on three-dimensional point cloud, in step S2, a straight-through filter is used to divide the area of ​​interest. Given a rectangular space of [x0,x1]×[y0,y1]×[z0,z1], when any point (x,y,z) is within the given range, that is, it meets x0≤x≤x1∩y0≤y≤y1∩z0≤z≤z1, the point is retained, otherwise the point is removed. The straight-through filter eliminates outliers that are not within the required range by limiting the range of the three-axis coordinates of the point cloud. The elimination speed is fast and can reduce the amount of subsequent calculations. Furthermore, the process of removing outliers from the point cloud is as follows. First, for each row of point cloud M in the three-dimensional point cloud model, search for each point m i The nearest neighbor of point m i and its k nearest neighbors m ij The average distance between i , Then calculate the average distance D and standard deviation σ of the point cloud M, K represents the number of points in the point cloud M; finally, determine point m i Whether it is an abnormal value, the judgment criteria are as follows: m o ={m i ∈M|d i ≤(D-σ·a)||d i ≥(D+σ·a)}, if point m i and its k neighboring points m ij If the average distance between the points exceeds a times the average distance of the point cloud, it can be considered as a noise point and removed; where a is a multiple of the standard deviation, m o is the point m that is judged as an outlier i The point cloud processed by Gaussian filter has a better smoothing effect.

[0012] Furthermore, in step S2, the specific extraction process of the solder joint data is as follows: input the preprocessed point cloud data, perform RANSAC straight line fitting on each row i of the point cloud to obtain the straight line model coe i , and project the point onto the straight line model coe i On; then calculate the straight line model coe iThe distance d between two adjacent points is taken as the distance between the upper and lower limits of the welding spacing. If the distance d is between the upper and lower limits of the welding spacing, the center point of the distance d is recorded as the welding point, and the data of this point is extracted. The RANSAC algorithm can robustly estimate the model parameters and estimate high-precision parameters from a data set containing a large number of outliers, so that the extracted welding point data can have higher accuracy.

[0013] Furthermore, in step S3, the welds obtained in step S2 are first sampled to generate an initial weld sequence, and then the weld sequence is sampled as control points to construct a NURBS curve that is similar to the control points, and the coordinate values ​​of the points on the NURBS curve are obtained; if the curve meets the requirements of the welding trajectory, it is used as the final welding trajectory, otherwise, new control points are sampled again from the initial weld sequence, and the node vectors are recalculated to construct a new NURBS curve until the curve meets the requirements of the welding trajectory, and the welding trajectory is used as the final welding trajectory. The NURBS algorithm is stable in calculation, fast in speed, and has an obvious geometric interpretation, so it can ensure the accuracy of welding trajectory extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a flow chart of a welding trajectory generation method based on three-dimensional point cloud of the present application;

[0015] Figure 2 is a schematic diagram of the sensor coordinate system and the welding robot base coordinate system in this application;

[0016] Figure 3 It is a schematic diagram of the solder joint data extraction process in step S2 of the present application. DETAILED DESCRIPTION

[0017] The present application is further described below in conjunction with the accompanying drawings and embodiments, but they are not intended to limit the present application. In the following embodiments, the contents not described in detail are all common technical knowledge in the art.

[0018] During the welding process, due to the influence of various factors, welds often have a variety of welding defects, such as undercuts, pits, weld nodules and other surface defects; and the current weld identification usually uses human eyes to detect the weld area of ​​the workpiece, which is inefficient and has low accuracy of detection results. In recent years, it has become more convenient to obtain 3D point cloud data using 3D scanners; automated geometric measurement systems can achieve efficient, accurate and consistent weld detection, significantly reduce human errors, and improve production efficiency and consistency. By matching the 3D point cloud with the template point cloud of known pose, the rigid transformation between the two can be obtained, thereby achieving accurate identification and positioning, making welding work intelligent and automated.

[0019] In view of this, the present application provides a method for generating a welding trajectory based on a three-dimensional point cloud; the method first uses a line structured light sensor to scan the weld of the weldment to effectively identify the weld and obtain a three-dimensional point cloud model of the weld; since the initial three-dimensional point cloud model obtained usually has redundancy and noise, these noises are not conducive to the extraction of subsequent features of the point cloud, so the present application uses a corresponding filtering algorithm to pre-process the point cloud in the model, remove outliers, and extract weld point data; finally, according to the trajectory of the extracted weld point data, a NURBS curve algorithm is used to generate a welding trajectory, so that the welding robot can perform welding operations according to actual conditions, that is, the welding robot can make corresponding welding process adjustments according to actual conditions, thereby improving the welding forming effect and work efficiency.

[0020] Example:

[0021] See also Figure 1 The welding trajectory generation method based on three-dimensional point cloud in this embodiment includes three steps: model establishment, feature extraction, and posture estimation; the specific process is as follows.

[0022] S1, model establishment:

[0023] a. Use a line structured light sensor to scan the weld area of ​​the weldment, treat each scanned data as a row of ordered point cloud and number them. At the same time, the scanning order is used as the column number of the ordered point cloud to obtain the ordered point cloud of the weld surface. After the point cloud is arranged in an orderly manner, it is convenient to establish the subsequent three-dimensional point cloud model of the weld.

[0024] b, the point cloud data N represented in the sensor coordinate system SEN , converted into point cloud data N represented in the welding robot base coordinate system BOT , realize the point cloud data N in the sensor coordinate system SEN A unified representation in the robot base coordinate system to obtain a 3D point cloud model of the weld; see Figure 2 ,Specifically, the conversion relationship between the sensor coordinate system and the welding robot base coordinate system is: Among them, N BOT Represents the point cloud data in the welding robot base coordinate system {BOT}; N SEN Represents point cloud data in the sensor coordinate system {SEN}; Represents the transformation matrix from the welding robot base coordinate system {BOT} to the intermediate coordinate system {BS}; Represents the transformation matrix from the sensor coordinate system {SEN} to the intermediate coordinate system {BS}.

[0025] S2, feature extraction:

[0026] a. Preprocess the point cloud in the three-dimensional point cloud model (in this application, a straight-through filter and a Gaussian filter are used to preprocess the point cloud, which is simple and fast, and has a good denoising effect, which is convenient for subsequent extraction of point cloud features):

[0027] a1, first use the straight-through filter to divide the area of ​​interest, and only retain the point cloud within the set coordinate system range; specifically, given a rectangular space of [x0,x1]×[y0,y1]×[z0,z1], when any point (x,y,z) is within the given range, that is, it meets x0≤x≤x1∩y0≤y≤y1∩z0≤z≤z1, retain the point, otherwise remove the point. The straight-through filter removes outliers that are not within the required range by limiting the range of the three-axis coordinates of the point cloud. The removal speed is fast and can reduce the amount of subsequent calculations.

[0028] a2, and then use Gaussian filter to remove outliers in the point cloud. The specific process is as follows: first, for each row of point cloud M in the 3D point cloud model, search for each point m i The nearest neighbor of point m i and its k nearest neighbors m ij The average distance d between i , Then calculate the average distance D and standard deviation σ of the point cloud M, K represents the number of points in the point cloud M; finally, determine point m i Whether it is an abnormal value, the judgment criteria are as follows: m o ={m i ∈M|d i ≤(D-σ·a)||d i ≥(D+σ·a)}, if point m i and its k neighboring points m ij If the average distance between the points exceeds a times the average distance of the point cloud, it can be considered as a noise point and removed; where a is a multiple of the standard deviation, m o is the point m that is judged as an outlier i The point cloud processed by Gaussian filter has a better smoothing effect.

[0029] b. See Figure 3 , input the preprocessed point cloud data, perform RANSAC straight line fitting on each row i of the point cloud to obtain the straight line model coe i , and project the point onto the straight line model coe i On; then calculate the straight line model coe iThe distance d between two adjacent points is taken as the distance between the upper and lower limits of the welding spacing (set according to the actual situation of different products), then the center point of the distance d is recorded as the welding point, and the data of this point is extracted. The RANSAC algorithm can robustly estimate the model parameters and estimate high-precision parameters from a data set containing a large number of outliers, so that the extracted welding point data can have high accuracy.

[0030] S3, pose estimation:

[0031] a. First, the weld points obtained in step S2 are sampled to generate an initial weld point sequence, and then the weld point sequence is sampled as control points, a NURBS curve approximating the control points is constructed, and the point coordinate values ​​on the NURBS curve are obtained; if the curve meets the requirements of the welding trajectory, it is used as the final welding trajectory; otherwise, new control points are sampled again from the initial weld point sequence, and the node vectors are recalculated to construct a new NURBS curve, until the curve meets the requirements of the welding trajectory, and the welding trajectory is used as the final welding trajectory;

[0032] b. Use Newton iteration method to optimize the final welding trajectory generated above to obtain the optimal welding trajectory (that is, the welding trajectory is infinitely close to the actual situation, thereby ensuring the welding quality and welding accuracy).

[0033] The above general description of the invention involved in this application and the description of its specific implementation methods should not be understood as limiting the technical solutions of the invention. Based on the disclosure of this application, those skilled in the art can add, reduce or combine the disclosed technical features in the above general description and / or specific implementation methods (including examples) without violating the constituent elements of the invention involved, to form other technical solutions within the scope of protection of this application.

Claims

1. A welding trajectory generation method based on three-dimensional point cloud, characterized in that: The method specifically comprises the following steps: S1, model building: First, use the line structured light sensor to scan the weld area of ​​the weldment to obtain the ordered point cloud of the weld surface; then, convert the point cloud data represented in the sensor coordinate system into the point cloud data represented in the welding robot base coordinate system to obtain the three-dimensional point cloud model of the weld; S2, feature extraction: pre-process the point cloud in the model, first use a straight-through filter to remove the point cloud outside the set coordinate system range, retain the point cloud within the set coordinate system range, and then use a Gaussian filter to remove the outliers of the point cloud; then, use the RANSAC algorithm to fit the pre-processed point cloud and extract the welding point data; S3, posture estimation: Based on the weld point data extracted in step S2, a NURBS curve algorithm is used to generate a welding trajectory.

2. The method for generating welding trajectories based on three-dimensional point clouds according to claim 1, characterized in that: In step S3, the generated welding trajectory is optimized using the Newton iteration method to obtain the optimal welding trajectory.

3. The welding trajectory generation method based on three-dimensional point cloud according to claim 1, characterized in that: In the step S1, the data of each scan is regarded as a row of ordered point cloud and numbered. At the same time, the scanning order is used as the column number of the ordered point cloud, so as to obtain the ordered point cloud of the weld surface.

4. The method for generating welding trajectories based on three-dimensional point clouds according to claim 3, characterized in that: In step S1, the conversion relationship between the sensor coordinate system and the welding robot base coordinate system is: N BOT Represents the point cloud data in the welding robot base coordinate system {BOT}, N SEN represents the point cloud data in the sensor coordinate system {SEN}, represents the transformation matrix from the welding robot base coordinate system {BOT} to the intermediate coordinate system {BS}, Represents the transformation matrix from the sensor coordinate system {SEN} to the intermediate coordinate system {BS}.

5. The method for generating welding trajectories based on three-dimensional point clouds according to claim 3, characterized in that: In step S2, a straight-through filter is used to divide the region of interest. Given a rectangular space of [x0,x1]×[y0,y1]×[z0,z1], when any point (x,y,z) is within the given range, that is, it satisfies x0≤x≤x1∩y0≤y≤y1∩z0≤z≤z1, the point is retained, otherwise the point is removed.

6. The method for generating welding trajectories based on three-dimensional point clouds according to claim 5, characterized in that: In step S2, the process of removing outliers in the point cloud is as follows: first, for each row of point cloud M in the three-dimensional point cloud model, search for each point m i The nearest neighbor of point m i and its k nearest neighbors m ij The average distance d between i , Then calculate the average distance D and standard deviation σ of the point cloud M, K represents the number of points in the point cloud M; finally, determine point m i Whether it is an abnormal value, the judgment criteria are as follows: m o ={m i ∈M|d i ≤(D-σ·a)||d i ≥(D+σ·a)}, if point m i and its k neighboring points m ij If the average distance between the points exceeds a times the average distance of the point cloud, it can be considered as a noise point and removed; where a is a multiple of the standard deviation, m o is the point m that is judged as an outlier i .

7. The method for generating welding trajectories based on three-dimensional point clouds according to claim 6, characterized in that: In step S2, the specific extraction process of the solder joint data is as follows: input the preprocessed point cloud data, perform RANSAC straight line fitting on each row i of the point cloud to obtain the straight line model coe i , and project the point onto the straight line model coe i On; then calculate the straight line model coe i The distance d between the two adjacent points is taken as the distance d between the upper and lower limits of the welding spacing, and if the distance d is between the upper and lower limits of the welding spacing, the center point of the distance d is recorded as the welding point and the data of the point is extracted.

8. The method for generating welding trajectories based on three-dimensional point clouds according to claim 7, characterized in that: In the step S3, the welds obtained in the step S2 are first sampled to generate an initial weld sequence, and then the weld sequence is sampled as control points to construct a NURBS curve approximating the control points, and the point coordinate values ​​on the NURBS curve are obtained; if the curve meets the requirements of the welding trajectory, it is used as the final welding trajectory, otherwise, new control points are resampled from the initial weld sequence, and the node vectors are recalculated to construct a new NURBS curve.

Images