Weld joint feature recognition and path extraction method based on line laser measurement

Abstract

The invention belongs to the technical field of three-dimensional measurement and weld joint processing, and discloses a weld joint feature recognition and path extraction method based on line laser measurement. According to the full-automatic welding seam shape extraction method, an automatic measurement part before welding seam polishing and other machining steps can be achieved, after a robot measures the position of a welding seam, welding seam shape extraction can be achieved, the position of the center line of the welding seam can be calculated, and therefore a machining path is obtained; and the robot can be directly guided to process the welding seam through the hand-eye matrix. In addition, the welding width and surplus height characteristics before and after welding seam machining can also be calculated and compared. After the welding width and the excess weld metal are calculated, the machining parameters can be further calculated, and the whole method provides technical support for automatic welding seam machining.

Description

Technical Field

[0001] The invention belongs to the technical field of three-dimensional measurement and weld processing, and in particular relates to a weld feature recognition and path extraction method based on line laser measurement. Background Art

[0002] With the gradual promotion of the intelligent manufacturing digital industry, industrial robots are increasingly used in engineering. Traditional processing projects need to be iterated in an automated way. For example, for weld grinding and welding tasks, the robot is required to find the weld position during the task. At the same time, after the robot holds the camera to shoot, the weld feature quantity calculation needs to be realized. The robot "obtains" the weld position, that is, it needs to obtain the coordinates of the weld relative to the robot base coordinate system. After the camera is measured, only the distance of the measured object relative to the camera measurement origin can be obtained, and the obtained three-dimensional point cloud is relative to the camera origin. Therefore, it is necessary to implement a hand-eye calibration experiment with sufficient accuracy to obtain the hand-eye calibration matrix, that is, to obtain the relative coordinates of the camera measurement origin relative to the robot flange. After obtaining the weld point cloud, it is necessary to extract the weld position and weld centerline, and calculate the weld width and excess height. After the weld centerline is extracted, the weld can be ground and other next processing, and the weld processing effect can be estimated based on the weld width and excess height. Therefore, it is a general trend for processing to move towards automation and intelligence to study the hand-eye calibration method that meets the accuracy standard and find the method of weld extraction and feature calculation.

[0003] Through the above analysis, the problems and defects of the prior art are as follows:

[0004] (1) Based on the method of obtaining weld information from the two-dimensional image of the weld, since the two-dimensional image lacks information of one dimension, it is difficult to obtain the height information of the weld and the relative position of the weld with respect to the robot.

[0005] (2) In the past, the methods for directly extracting welds and calculating their features through point clouds were mostly methods that combined two-dimensional image features, which made it difficult to directly analyze point clouds to obtain weld features.

[0006] (3) Previous processing methods based on weld point clouds did not combine the robot posture to realize weld extraction. They could not directly obtain the position coordinates of the extracted weld relative to the robot, nor could they combine the weld surface to obtain the relative posture of the robot during processing.

[0007] The difficulty of solving the above problems and defects is:

[0008] A weld extraction method and a weld feature calculation method based on the local geometric features of point clouds are proposed, which can directly obtain the three-dimensional information of the weld. The hand-eye calibration method can be used to obtain the relative position of the weld with respect to the robot, which can realize fully automatic recognition of the weld and make direct preparations for subsequent processing. Summary of the invention

[0009] In view of the problems existing in the prior art, the present invention provides a weld feature recognition and path extraction method based on line laser measurement.

[0010] The present invention is implemented in this way: a weld feature recognition and path extraction method based on line laser measurement includes:

[0011] S1. Use the hand-eye calibration experiment to calibrate the hand-eye matrix of the robot, and obtain the coordinate transformation matrix of the origin of the measurement point cloud of the line laser camera relative to the origin of the robot flange;

[0012] S2. Use the controller module to synchronously record the line laser camera scanning point cloud data and the terminal coordinates and posture of the robot relative to the base coordinate system, and reconstruct the weld scanning point cloud according to the calculated hand-eye matrix, robot motion data, and line laser scanning data. The coordinate system of the reconstructed point cloud is the robot base coordinate system.

[0013] S3. Based on the reconstructed weld point cloud, preprocessing for subsequent point cloud local feature calculation is performed. The weld position is extracted by using a clustering method based on the calculated local geometric features of the point cloud. The plane where the weld is located is fitted, and the weld centerline is obtained by extracting the highest point on the weld from the plane.

[0014] S4. According to the extracted weld, the PCA algorithm is used to calculate the vector of the main direction of the weld, which is called the centerline vector, and the plane passing through the highest point of the weld and perpendicular to the centerline vector is the centerline vertical plane; for each single laser line, find the point farthest from the center point of the weld to the left and right to obtain the edge point of the weld; on the same laser line, calculate the distance between the two points on the edge of the weld, calculate the projection length of the line segment between the two points on the center line of the weld, and calculate the projection length of the line segment length on the centerline vertical plane; the projection length of the line segment length on the centerline vertical plane is considered to be the weld width, and the distance between the point on the center line of the weld and the fitting plane of the plate where the weld is located is the weld height.

[0015] Further, in step S1, by formula A i XC i =p i Establish an overdetermined linear equation and solve the hand-eye matrix X, where A i is the robot pose, c i is the point cloud position captured by the camera, p i is the position of the same point in the point cloud in the robot's base system;

[0016] In the specific experiment, the standard ball is fixed on the turntable, and the robot measures the cross section of the standard ball in multiple postures; according to the cross section of the standard ball, the center of the circular cross section is fitted by the least square method, and the coordinates of the center of the ball relative to the camera are calculated according to the nominal diameter of the standard ball;

[0017]

[0018] x circle 、z circle is the point cloud position on the circular cross section obtained by measurement, where x ball ,y ball 、z ball is the calculated coordinate of the sphere center relative to the camera, R is the standard radius of the standard sphere, and r is the cross-sectional radius calculated from the circular cross section.

[0019] Further, according to the robot posture homogeneous matrix A i , unknown hand-eye matrix X, the center position of the ball in the camera coordinate system The center position of the ball in the robot base coordinate system Available The method for obtaining the hand-eye calibration equation, based on the principle that the center position of the standard ball remains unchanged, can obtain the following equation according to the measurement data:

[0020]

[0021] Among them, at the i-th measurement, The robot's i-th posture homogeneous transformation matrix A i Using the rotation matrix and translation vectors express; The hand-eye calibration homogeneous transformation matrix X is transformed by the rotation matrix R X and the translation vector t X indicates;

[0022] Furthermore, the overdetermined linear equation composed of the above connected equations can be solved by the least squares method to obtain the hand-eye matrix X.

[0023] Further, in step S2, after the hand-eye calibration experiment, the camera scans the weld to be detected, and obtains the reconstructed point cloud of the weld relative to the robot base coordinates through the robot posture, hand-eye matrix, and camera measurement point cloud; the robot posture measurement data and camera measurement data are processed, combined with the hand-eye matrix X, and the formula A is used i XC i =p i , reconstruct the weld scanning point cloud.

[0024] Furthermore, in step S3, for the reconstruction of the weld point cloud, the reconstructed point cloud needs to be preprocessed first; the uneven distribution of the weld point cloud may be uneven in the horizontal and vertical directions due to the uneven movement speed of the robot, and the uneven distribution of the point cloud may lead to large errors in the calculation of local normal quantities and curvatures after the kd-tree nearest neighbor search in the subsequent processing; grid filtering and statistical filtering are used to obtain the downsampled point cloud; the least squares re-fitting is performed on the point cloud surface to generate a smooth surface;

[0025] Use kd-tree to store point cloud data and record the neighboring points of each point in the point cloud. For the point cloud surface points and their neighboring point cloud clusters, use the eigenvalues ​​of the local point cloud cluster covariance matrix to further calculate the surface normal. The covariance matrix of a point and its nearby point cloud is:

[0026]

[0027] Among them, p i is the point and its surrounding points, is the centroid of the point set;

[0028] Perform eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors. The eigenvector corresponding to the maximum eigenvalue is the main direction of the point cloud, indicating the main direction of the point cloud extension. The eigenvector corresponding to the minimum eigenvalue is the normal direction of the point cloud surface, indicating that the point cloud has the smallest change in this direction. The surface curvature is estimated by the eigenvalue of the covariance matrix and the surface curvature is as follows:

[0029]

[0030] Among them, λ0 is the minimum value among the three eigenvalues;

[0031] The region growing algorithm sorts the points according to the curvature value of the point cloud; the region growth starts from the point with the smallest curvature, and the selected point is added to the set of seed points; when making the neighborhood set of the seed point, when the angle between the normal of the adjacent point and the normal of the seed point is less than the threshold, and the curvature value of the neighborhood point of the point is less than the threshold, the adjacent point is added to the set of seed points; then the current seed point is removed from the seed list, and the steps of finding the seed point set are repeated until the seed list is empty and the region growing algorithm is completed;

[0032] When extracting the weld centerline, the planes on both sides of the weld are fitted as a whole, the distance from the weld point cloud to the plane is calculated, the highest point of the weld is found, and the highest point of the weld is fitted as a straight line, which is the weld centerline.

[0033] Further, in step S4, for the weld extracted in S3, the PCA algorithm is used to calculate the main orientation of the weld to obtain the weld orientation vector; since the point cloud obtained by scanning with the line laser sensor is arranged in rows, the weld center point of each row has been extracted in S3, the farthest point on the left and right sides of the extracted weld from the center point is found as the weld edge point, and the distance between the left and right points of the center point is calculated, which is recorded as l;

[0034] According to the main orientation vector of the weld and the center point of the weld, calculate the plane passing through the center point of the weld and with the main orientation vector of the weld as the normal, and record it as the center line perpendicular plane; calculate the projection length of the distance between the two center points on the main orientation vector and the projection line segment length on the center line perpendicular plane respectively; the projection line segment length on the center line perpendicular plane is the weld width.

[0035] Another object of the present invention is to provide a weld feature recognition and path extraction system based on line laser measurement, comprising:

[0036] The calibration module is used to calibrate the hand-eye matrix of the robot using the hand-eye calibration experiment to obtain the coordinate transformation matrix of the origin of the measurement point cloud of the line laser camera relative to the origin of the robot flange;

[0037] The controller module is used to synchronously record the line laser camera scanning point cloud data and the terminal coordinates and posture of the robot relative to the base coordinate system using the controller module, and reconstruct the weld scanning point cloud according to the calculated hand-eye matrix, robot motion data, and line laser scanning data. The coordinate system of the reconstructed point cloud is the robot base coordinate system;

[0038] The reconstruction module is used to perform preprocessing for subsequent point cloud local feature calculation based on the reconstructed weld point cloud, extract the weld position based on the calculated point cloud local geometric features and use the clustering method; fit the plane where the weld is located, and obtain the weld centerline by extracting the highest point on the weld from the plane;

[0039] The calculation module is used to calculate the vector of the main direction of the weld using the PCA algorithm based on the extracted weld, which is called the centerline vector, and the plane passing through the highest point of the weld and perpendicular to the centerline vector is the centerline vertical plane; for each single laser line, find the point farthest from the center point of the weld to the left and right to obtain the edge point of the weld; on the same laser line, calculate the distance between two points on the edge of the weld, calculate the projection length of the line segment length between the two points on the center line of the weld, and calculate the projection length of the line segment length on the centerline vertical plane; the projection length of the line segment length on the centerline vertical plane is considered to be the weld width, and the distance between the point on the center line of the weld and the fitting plane of the plate where the weld is located is the weld height.

[0040] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the weld feature recognition and path extraction method based on line laser measurement.

[0041] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the weld feature recognition and path extraction method based on line laser measurement.

[0042] Another object of the present invention is to provide an information data processing terminal, which is used to implement the weld feature recognition and path extraction system based on line laser measurement.

[0043] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0044] First, the present invention proposes a fully automated method for weld morphology extraction, which can realize the automated measurement part before the weld grinding and other processing steps, so that after the robot measures the weld position, it can realize weld morphology extraction and calculate the weld centerline position to obtain the processing path. At the same time, the hand-eye matrix can directly guide the robot to process the weld. In addition, the weld width and excess height characteristics before and after the weld processing can also be calculated and compared. After the weld width and excess height calculation are realized, the processing parameters can be further calculated. The whole method provides technical support for the automated processing of welds.

[0045] Second, as auxiliary evidence of the inventiveness of the claims of the present invention, it is also reflected in the following important aspects:

[0046] (1) The expected benefits and commercial value of the technical solution of the present invention after transformation are:

[0047] By scanning the weld with a line laser sensor and designing a point cloud processing algorithm based on the weld features, combined with hand-eye calibration, the weld processing can be automated, which greatly reduces manual intervention and reduces labor costs. At the same time, robot operation reduces human operating errors and improves the stability of the production line and the consistency of weld processing. In addition, combined with real-time scanning and point cloud analysis, the entire welding process can be automatically monitored, and any welds that do not meet the standards can be found in time, and automatic corrections can be made to reduce rework and defective rates; recording and storing real-time data of weld scanning and grinding processes can provide rich digital support for quality monitoring, production scheduling and post-analysis, and further optimize welding and grinding processes. In small-batch production, automated grinding technology can quickly switch between different workpieces, improving production flexibility.

[0048] (2) The technical solution of the present invention fills the technical gap in the industry at home and abroad:

[0049] Traditional weld grinding relies on manual operation, but manual operation has low precision and insufficient stability, which easily leads to uneven grinding or damage to the weld. In recent years, many scholars at home and abroad have studied the automated welding and grinding technology of welds, but there is no detailed three-dimensional point cloud processing algorithm for different types of butt welds. For planar butt welds, many methods for obtaining weld positions based on depth cameras have been proposed, but this method based on the combination of two-dimensional image processing and depth map has low precision, is easily affected by ambient light, and cannot effectively process complex geometric shapes. The method for obtaining point clouds based on line laser sensors proposed in the present invention integrates multiple steps such as robot hand-eye calibration, line laser scanning, point cloud reconstruction and analysis, and machine learning, which can realize a complete automated processing closed loop. Point cloud scanning and analysis are completed automatically by the system, which improves efficiency and accuracy. Combined with the weld point cloud data calculated along the length of the weld, the weld width and excess height, the automated grinding system can provide real-time feedback on the state of the weld, and automatically plan the grinding path and strength according to the feedback, thereby realizing dynamic adaptive control and providing a more accurate grinding effect.

[0050] Third, at present, in the welding manufacturing industry, there are still many technical difficulties in weld feature recognition and path extraction technology. On the one hand, traditional weld feature recognition methods rely on manual calibration or simple two-dimensional image processing technology, which makes it difficult to achieve three-dimensional reconstruction and accurate feature extraction of welds. On the other hand, existing weld path planning methods do not perform well in complex welding scenarios and cannot effectively cope with changes in weld geometry, such as dynamic adjustment of weld width, height and direction. In addition, noise interference of point cloud data and insufficient accuracy in the multi-sensor data fusion process further limit the reliability of weld recognition and path extraction.

[0051] The present invention proposes a weld feature recognition and path extraction method based on local geometric features. The accurate spatial mapping of the line laser camera and the robot system is achieved through hand-eye calibration, ensuring the high-precision synchronization of the point cloud data and the robot's position information. Through the preprocessing of point cloud data and local feature analysis, the clustering algorithm and principal component analysis (PCA) method are used to efficiently extract geometric features such as weld position, width and height, significantly improving the accuracy of weld recognition. At the same time, the dynamically adjusted weld path planning strategy effectively solves the problem of complex and changeable weld geometry.

[0052] The present invention can be widely used in the fields of shipbuilding, automobile manufacturing, building steel structure, aerospace, etc., and provides reliable technical support for high-precision weld identification and welding path planning. By realizing the efficient coordination of line laser scanning and robot system, the present invention greatly reduces the need for manual intervention in the welding process and improves the automation level and production efficiency of the welding process. In addition, based on the precise extraction of weld features, the present invention can significantly reduce the incidence of welding defects and improve the quality and reliability of products.

[0053] By introducing advanced line laser scanning and dynamic algorithm optimization technology, the present invention has achieved a technological breakthrough in the field of welding path planning and welding automation, solving the problems of insufficient precision and low efficiency in traditional methods. The industrial application of this technology can not only significantly improve the intelligence level of welding production lines, but also provide strong technical support for the digital upgrade of the manufacturing industry, and provide a new solution for enterprises to reduce production costs and improve market competitiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flow chart of a weld feature recognition and path extraction method based on line laser measurement provided by an embodiment of the present invention.

[0055] Figure 2 It is a structural block diagram of a weld feature recognition and path extraction system based on line laser measurement provided by an embodiment of the present invention.

[0056] Figure 3 It is a layout diagram of the experimental equipment provided by the embodiment of the present invention.

[0057] Figure 4 This is the original weld image provided by the embodiment of the present invention.

[0058] Figure 5 It is a weld centerline and edge extraction diagram provided by an embodiment of the present invention.

[0059] Figure 6 It is a schematic diagram of weld centerline extraction provided by an embodiment of the present invention.

[0060] Figure 7 It is a distribution diagram of the weld width along the y-axis of the weld provided by an embodiment of the present invention.

[0061] Figure 8 It is a distribution diagram of weld height along the y-axis length of the weld provided by an embodiment of the present invention.

[0062] Fig. 9 This is a diagram showing the extraction of edge points and center points of a flat plate weld before and after grinding provided by an embodiment of the present invention.

[0063] Fig.10 It is a distribution diagram of weld height and weld width along the y-axis length of a ground weld provided by an embodiment of the present invention.

[0064] Fig.11 This is Example 2 provided in an embodiment of the present invention: a schematic diagram of angle butt weld extraction and edge point extraction. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0066] like Figure 1 As shown, a weld feature recognition and path extraction method based on line laser measurement provided by an embodiment of the present invention includes the following steps:

[0067] S1. Use the hand-eye calibration experiment to calibrate the hand-eye matrix of the robot, and obtain the coordinate transformation matrix of the origin of the measurement point cloud of the line laser camera relative to the origin of the robot flange;

[0068] S2. Use the controller module to synchronously record the line laser camera scanning point cloud data and the terminal coordinates and posture of the robot relative to the base coordinate system, and reconstruct the weld scanning point cloud according to the calculated hand-eye matrix, robot motion data, and line laser scanning data. The coordinate system of the reconstructed point cloud is the robot base coordinate system.

[0069] S3. Based on the reconstructed weld point cloud, preprocessing for subsequent point cloud local feature calculation is performed. The weld position is extracted by using a clustering method based on the calculated local geometric features of the point cloud. The plane where the weld is located is fitted, and the weld centerline is obtained by extracting the highest point on the weld from the plane.

[0070] S4. According to the extracted weld, the PCA algorithm is used to calculate the vector of the main direction of the weld, which is called the centerline vector, and the plane passing through the highest point of the weld and perpendicular to the centerline vector is the centerline vertical plane; for each single laser line, find the point farthest from the center point of the weld to the left and right to obtain the edge point of the weld; on the same laser line, calculate the distance between the two points on the edge of the weld, calculate the projection length of the line segment between the two points on the center line of the weld, and calculate the projection length of the line segment length on the centerline vertical plane; the projection length of the line segment length on the centerline vertical plane is considered to be the weld width, and the distance between the point on the center line of the weld and the fitting plane of the plate where the weld is located is the weld height.

[0071] In step S1 provided in the embodiment of the present invention, by formula A i XC i =p i Establish an overdetermined linear equation and solve the hand-eye matrix X, where A i is the robot pose, c iis the point cloud position captured by the camera, p i is the position of the same point in the point cloud in the robot's base system;

[0072] In the specific experiment, the standard ball is fixed on the turntable, and the robot measures the cross section of the standard ball in multiple postures; according to the cross section of the standard ball, the center of the circular cross section is fitted by the least square method, and the coordinates of the center of the ball relative to the camera are calculated according to the nominal diameter of the standard ball;

[0073]

[0074] x circle 、z circle is the point cloud position on the circular cross section obtained by measurement, where x ball ,y ball 、z ball is the calculated coordinate of the sphere center relative to the camera, R is the standard radius of the standard sphere, and r is the cross-sectional radius calculated from the circular cross section.

[0075] According to the robot posture homogeneous matrix A provided in the embodiment of the present invention i , unknown hand-eye matrix X, the center position of the ball in the camera coordinate system The center position of the ball in the robot base coordinate system Available The method for obtaining the hand-eye calibration equation, based on the principle that the center position of the standard ball remains unchanged, can obtain the following equation according to the measurement data:

[0076]

[0077] Among them, at the i-th measurement, The robot's i-th posture homogeneous transformation matrix A i Using the rotation matrix and translation vectors express; The hand-eye calibration homogeneous transformation matrix X is transformed by the rotation matrix R X and the translation vector t X indicates;

[0078] Furthermore, the overdetermined linear equation composed of the above connected equations can be solved by the least squares method to obtain the hand-eye matrix X.

[0079] In step S2 provided by the embodiment of the present invention, after the hand-eye calibration experiment, the camera scans the weld to be detected, and obtains the reconstructed point cloud of the weld relative to the robot base coordinates through the robot posture, hand-eye matrix, and camera measurement point cloud; the robot posture measurement data and camera measurement data are processed, combined with the hand-eye matrix X, and the formula A is used i XC i =p i, reconstruct the weld scanning point cloud.

[0080] In step S3 provided by the embodiment of the present invention, for the reconstruction of the weld point cloud, the reconstructed point cloud needs to be preprocessed first; the uneven distribution of the weld point cloud in the horizontal and vertical directions may be caused by the uneven movement speed of the robot, and the uneven distribution of the point cloud may cause large errors when calculating the local normal quantity and curvature after the kd-tree nearest neighbor search in the subsequent processing; grid filtering and statistical filtering are used to obtain the downsampled point cloud; the least squares re-fitting is performed on the point cloud surface to generate a smooth surface;

[0081] Use kd-tree to store point cloud data and record the neighboring points of each point in the point cloud. For the point cloud surface points and their neighboring point cloud clusters, use the eigenvalues ​​of the local point cloud cluster covariance matrix to further calculate the surface normal. The covariance matrix of a point and its nearby point cloud is:

[0082]

[0083] Among them, p i is the point and its surrounding points, is the centroid of the point set;

[0084] Perform eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors. The eigenvector corresponding to the maximum eigenvalue is the main direction of the point cloud, indicating the main direction of the point cloud extension. The eigenvector corresponding to the minimum eigenvalue is the normal direction of the point cloud surface, indicating that the point cloud has the smallest change in this direction. The surface curvature is estimated by the eigenvalue of the covariance matrix and the surface curvature is as follows:

[0085]

[0086] Among them, λ0 is the minimum value among the three eigenvalues;

[0087] The region growing algorithm sorts the points according to the curvature value of the point cloud; the region growth starts from the point with the smallest curvature, and the selected point is added to the set of seed points; when making the neighborhood set of the seed point, when the angle between the normal of the adjacent point and the normal of the seed point is less than the threshold, and the curvature value of the neighborhood point of the point is less than the threshold, the adjacent point is added to the set of seed points; then the current seed point is removed from the seed list, and the steps of finding the seed point set are repeated until the seed list is empty and the region growing algorithm is completed;

[0088] When extracting the weld centerline, the planes on both sides of the weld are fitted as a whole, the distance from the weld point cloud to the plane is calculated, the highest point of the weld is found, and the highest point of the weld is fitted as a straight line, which is the weld centerline.

[0089] In step S4 provided by the embodiment of the present invention, for the weld extracted in S3, the PCA algorithm is used to calculate the main orientation of the weld to obtain the weld orientation direction vector; since the point cloud obtained by scanning the line laser sensor is arranged in rows, the weld center point of each row has been extracted in S3, the farthest point on the left and right sides of the extracted weld from the center point is found as the weld edge point, and the distance between the left and right points of the center point is calculated, which is recorded as l;

[0090] According to the main orientation vector of the weld and the center point of the weld, calculate the plane passing through the center point of the weld and with the main orientation vector of the weld as the normal, and record it as the center line perpendicular plane; calculate the projection length of the distance between the two center points on the main orientation vector and the projection line segment length on the center line perpendicular plane respectively; the projection line segment length on the center line perpendicular plane is the weld width.

[0091] Through the hand-eye calibration experiment, the spatial relationship between the coordinates of the end of the robot arm and the point cloud origin installed on the line laser camera is determined. The goal of this step is to obtain a hand-eye matrix, which represents the coordinate transformation relationship between the origin of the measured point cloud of the line laser camera and the origin of the robot flange. Through the precise calculation of the calibration experiment, the mapping of laser scanning point cloud data and robot motion data in a unified coordinate system can be achieved.

[0092] The controller module is used to synchronously record the point cloud data collected by the line laser camera and the end position and posture data relative to the base coordinate system during the robot's motion. Through the calibrated hand-eye matrix, the point cloud data obtained by the line laser scanning is converted to the robot's base coordinate system and combined with the robot's motion data to complete the three-dimensional reconstruction of the weld scanning point cloud. This process ensures that the reconstructed point cloud data has high-precision coordinate information in space.

[0093] The reconstructed weld point cloud is preprocessed, including denoising, point cloud smoothing and other operations to ensure the continuity and accuracy of the data. Based on the preprocessed point cloud data, the local geometric features of the point cloud, such as the normal vector and curvature, are calculated, and the location of the weld is extracted in combination with the clustering algorithm. This step accurately locates the weld centerline by extracting the local highest point of the weld point cloud.

[0094] Fit the highest point set of the weld point cloud, extract the plane where the weld is located, and obtain the center line of the weld. The accuracy of the fitted plane determines the position accuracy of the weld center line. The weld center line is the core of the welding path, and the accuracy of subsequent operations depends on the extraction effect of the center line.

[0095] Based on the centerline and the highest point of the weld, the principal component analysis (PCA) method is used to calculate the main orientation of the weld and determine the weld centerline vector. By establishing a plane perpendicular to the centerline vector, that is, the centerline perpendicular plane, the edge points on both sides of the weld are analyzed. In each laser line, the two side points farthest from the weld center point are found as the weld edge points, and their geometric relationship is calculated.

[0096] For each laser line, the weld width is defined as the projection length of the weld edge point on the vertical plane of the center line; the weld height is defined as the vertical distance from the highest point of the weld to the fitting plane of the weld plane. Combined with the overall data of the point cloud, the geometric size characteristics of the weld are generated, providing a reliable data basis for subsequent welding path planning. Ultimately, the accurate extraction of weld feature parameters and path optimization are achieved.

[0097] like Figure 2 As shown, a weld feature recognition and path extraction system based on line laser measurement provided by an embodiment of the present invention includes:

[0098] The calibration module is used to calibrate the hand-eye matrix of the robot using the hand-eye calibration experiment to obtain the coordinate transformation matrix of the origin of the measurement point cloud of the line laser camera relative to the origin of the robot flange;

[0099] The controller module is used to synchronously record the line laser camera scanning point cloud data and the terminal coordinates and posture of the robot relative to the base coordinate system using the controller module, and reconstruct the weld scanning point cloud according to the calculated hand-eye matrix, robot motion data, and line laser scanning data. The coordinate system of the reconstructed point cloud is the robot base coordinate system;

[0100] The reconstruction module is used to perform preprocessing for subsequent point cloud local feature calculation based on the reconstructed weld point cloud, extract the weld position based on the calculated point cloud local geometric features and use the clustering method; fit the plane where the weld is located, and obtain the weld centerline by extracting the highest point on the weld from the plane;

[0101] The calculation module is used to calculate the vector of the main direction of the weld using the PCA algorithm based on the extracted weld, which is called the centerline vector, and the plane passing through the highest point of the weld and perpendicular to the centerline vector is the centerline vertical plane; for each single laser line, find the point farthest from the center point of the weld to the left and right to obtain the edge point of the weld; on the same laser line, calculate the distance between two points on the edge of the weld, calculate the projection length of the line segment length between the two points on the center line of the weld, and calculate the projection length of the line segment length on the centerline vertical plane; the projection length of the line segment length on the centerline vertical plane is considered to be the weld width, and the distance between the point on the center line of the weld and the fitting plane of the plate where the weld is located is the weld height.

[0102] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the weld feature recognition and path extraction method based on line laser measurement.

[0103] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the weld feature recognition and path extraction method based on line laser measurement.

[0104] Another object of the present invention is to provide an information data processing terminal, which is used to implement the weld feature recognition and path extraction system based on line laser measurement.

[0105] The present invention is specifically implemented:

[0106] like Figure 3 The present invention uses an ABB robot, a Beckhoff controller, a Keyence LJ-X8080 line laser camera, a Keyence camera controller, and an EL2262 terminal to build a measurement hardware platform. The Beckhoff controller automatically reads the robot's posture, and the EL2262 terminal sends a start signal to simultaneously trigger the reading of the robot's posture and the line laser camera's point cloud data.

[0107] Conduct a hand-eye calibration experiment, collect 20 sets of standard sphere data for the standard sphere section, ensure that the line laser is always located on one side of the sphere center, and record the robot posture data and the corresponding line laser measurement point cloud data. Obtain the hand-eye calibration matrix according to the algorithm. The standard sphere section circle measured by the line laser camera is used to perform the least squares fitting algorithm on the section point cloud to obtain the section circle radius and the center position. Combined with the standard sphere radius, the x, y, and z coordinates of the standard sphere center are obtained. The x is obtained by the following formula 1: ball ,y ball 、z ball :

[0108]

[0109] where x ball ,y ball 、z ball is the calculated coordinate of the sphere center relative to the camera, x circle 、z circle is the center position of the circle on the measured circular section, R is the standard radius of the standard sphere, and r is the cross-sectional radius calculated from the circular section.

[0110] Assume that the robot pose homogeneous matrix is ​​A i , the unknown hand-eye matrix is ​​X, and the center of the ball in the camera coordinate system is

[0111] Set to The position of the ball center in the robot base coordinate system is Available The method for obtaining the hand-eye calibration equation, based on the principle that the center position of the standard ball remains unchanged, can obtain equation 2 according to the measurement data:

[0112]

[0113] Among them, at the i-th measurement, The robot's i-th posture homogeneous transformation matrix A i Using the rotation matrix and translation vectors express. The hand-eye calibration homogeneous transformation matrix X is transformed by the rotation matrix R X and the translation vector t X Indicates. By solving the overdetermined linear equation composed of the above connected equations by the least squares method, the desired hand-eye matrix X can be obtained.

[0114] like Figure 4 , Figure 5 After the hand-eye calibration experiment, the weld measurement is performed. When the robot motion posture and camera measurement information are collected synchronously, the EL2262 terminal sends a continuous trigger signal, and the host computer controls the camera through code to batch record the collected data under external triggering, and records the robot motion posture data read by the Beckhoff controller. Fix the position of the weld to be measured, measure the weld, and reconstruct the weld point cloud according to the hand-eye matrix.

[0115] like Figure 6 , Figure 7 The regional growing algorithm is used for the weld point cloud, and the weld position is further extracted by Gaussian curvature. When calculating the local features of the weld, such as curvature and normal vector, the kd-tree is used to search the point cloud neighbors, and the k neighbor point sets of each point in the point cloud are recorded. The principal component analysis (PCA) method is used to further calculate the local neighbor point set point cloud cluster to calculate the point cloud surface normal vector and curvature. Record p i is the i-th point on the point cloud, is the centroid of the set of m neighboring points of the i-th point. The covariance matrix of a point and its nearby point cloud is shown in formula 3:

[0116]

[0117] Perform eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors. The eigenvector corresponding to the maximum eigenvalue is the main direction of the point cloud, indicating the main direction of the point cloud extension. The eigenvector corresponding to the minimum eigenvalue is the normal direction of the point cloud surface, indicating that the point cloud has the smallest change in this direction. Let λ0 be the minimum of the three eigenvalues, and λ1 and λ2 are the other two eigenvalues. Formula 4 estimates the i-th point p according to the eigenvalue i The surface curvature cur at:

[0118]

[0119] like Figure 8 , Fig. 9 When implementing the region growing algorithm, the point cloud is sorted according to its curvature value so that the region growing algorithm starts to grow from the point with the smallest curvature, and the selected point is added to the set where the seed point is located. When making the neighborhood set of seed points, when the angle between the normal of the adjacent point and the normal of the seed, and the curvature value of the neighborhood point are less than the threshold, the adjacent point is added to the set of seed points. The next step is to remove the current seed point from the seed list. The cycle includes two steps, seed point expansion and current seed point removal, until the seed list is empty and the region growing algorithm is completed. After removing the plane according to the region growing algorithm, the PCL point cloud library is used to calculate the Gaussian curvature of the point cloud, and the threshold is set to extract the complete weld part.

[0120] The principal component analysis (PCA) method is used to solve the main orientation vector of the weld as a whole, which is recorded as Perform plane fitting on the plane separated in the previous article and calculate the plane normal vector. Calculate the distance from the point on the weld to the plane, find the highest point of the weld, and obtain the weld residual height distributed along the weld direction on the weld. According to the main direction of the weld and the highest point of the weld, get the estimated weld centerline. Since the line laser scanning point cloud is an ordered point cloud sorted by row, search and extract the farthest points on both sides of the weld center point, recognize them as the weld edge points, and calculate the distance between the two points, recorded as l.

[0121] According to the main orientation vector of the weld and the center point of the weld, calculate the plane passing through the center point of the weld and with the main orientation vector of the weld as the normal, and record it as the center line perpendicular plane. Calculate the projection length of the distance l between the two points on the main orientation vector, record it as l1; at the same time, calculate the length of the projection line segment of the distance between the two points on the center line perpendicular plane, record it as l2. The modulus length of l1 is obtained from formula 5, and the modulus length of l2 is obtained from formula 6. The modulus length of l2 is the weld width.

[0122]

[0123]

[0124] The present invention relates to the field of scanning a weld seam by a robot holding a line laser camera, reconstructing a weld seam point cloud, and a point cloud processing algorithm, and the relevant technologies can be applied in multiple technical fields. In the field of automated welding and weld quality detection, the entire invention process can be used for automated welding quality control, especially for the accurate detection and evaluation of complex welded joints such as planar butt joints, corner joints, etc. In the process of welding process development and optimization, the welding parameters can be optimized by reconstructing and analyzing different weld seam shapes, thereby improving welding quality and production efficiency. In addition, in many complex engineering projects, the design files of parts may be lost or incomplete. By scanning with a robot holding a line laser camera, the three-dimensional geometric data of the workpiece can be accurately obtained, and a complete three-dimensional model can be generated by point cloud reconstruction technology for subsequent design and analysis. In the application field of the manufacturing industry, the welding, grinding and part assembly of automobile and aerospace structures are one of the key processing technologies. This system, which integrates robots, line laser cameras, point cloud processing software and intelligent algorithms, can automatically detect the quality of welded joints, including the height, width and shape of welds, and can ensure the quality of welds.

[0125] The present invention provides two local weld processing effect diagrams for flat plate welds and corner butt welds, and extracts weld edge points and center points for flat plate welds, and provides effect diagrams of the changes in weld width and excess height along the weld length. In the effect diagram of weld edge point and center point extraction, a schematic diagram of calculating the weld width from the distance between weld edge points is given. Before and after the plane weld grinding experiment, it can be observed that the weld height is reduced and the weld width remains basically unchanged. When processing the complex butt weld in the example diagram, the weld and the corners of the two edge plates are segmented by clustering to obtain the weld position, and the weld extraction edge points are displayed to indicate the conversion of the distance between the two edge points of the weld to the weld width.

[0126] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0127] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A weld feature recognition and path extraction method based on line laser measurement, characterized in that: The following steps are involved: S1. Use the hand-eye calibration experiment to calibrate the hand-eye matrix of the robot, and obtain the coordinate transformation matrix of the origin of the measurement point cloud of the line laser camera relative to the origin of the robot flange; S2. Use the controller module to synchronously record the line laser camera scanning point cloud data and the terminal coordinates and posture of the robot relative to the base coordinate system, and reconstruct the weld scanning point cloud according to the calculated hand-eye matrix, robot motion data, and line laser scanning data. The coordinate system of the reconstructed point cloud is the robot base coordinate system. S3. Based on the reconstructed weld point cloud, preprocessing for subsequent point cloud local feature calculation is performed. The weld position is extracted by using a clustering method based on the calculated local geometric features of the point cloud. The plane where the weld is located is fitted, and the weld centerline is obtained by extracting the highest point on the weld from the plane. S4. According to the extracted weld, the PCA algorithm is used to calculate the vector of the main direction of the weld, which is called the centerline vector, and the plane passing through the highest point of the weld and perpendicular to the centerline vector is the centerline vertical plane; for each single laser line, find the point farthest from the center point of the weld to the left and right to obtain the edge point of the weld; on the same laser line, calculate the distance between the two points on the edge of the weld, calculate the projection length of the line segment between the two points on the center line of the weld, and calculate the projection length of the line segment length on the centerline vertical plane; the projection length of the line segment length on the centerline vertical plane is considered to be the weld width, and the distance between the point on the center line of the weld and the fitting plane of the plate where the weld is located is the weld height.

2. The weld feature recognition and path extraction method based on line laser measurement as claimed in claim 1, characterized in that: In step S1, by formula A i XC i =p i Establish an overdetermined linear equation and solve the hand-eye matrix X, where A i is the robot pose, c i is the point cloud position captured by the camera, p i is the position of the same point in the point cloud in the robot's base system; In the specific experiment, the standard ball is fixed on the turntable, and the robot measures the cross section of the standard ball in multiple postures; according to the cross section of the standard ball, the center of the circular cross section is fitted by the least square method, and the coordinates of the center of the ball relative to the camera are calculated according to the nominal diameter of the standard ball; x circle 、z circle is the point cloud position on the circular cross section obtained by measurement, where x ball ,y ball 、z ball is the calculated coordinate of the sphere center relative to the camera, R is the standard radius of the standard sphere, and r is the cross-sectional radius calculated from the circular cross section.

3. The weld feature recognition and path extraction method based on line laser measurement as claimed in claim 1, characterized in that: According to the robot pose homogeneous matrix A i , unknown hand-eye matrix X, the center position of the ball in the camera coordinate system The center position of the ball in the robot base coordinate system Available The method for obtaining the hand-eye calibration equation, based on the principle that the center position of the standard ball remains unchanged, can obtain the following equation according to the measurement data: Among them, at the i-th measurement, The robot's i-th posture homogeneous transformation matrix A i Using the rotation matrix and translation vectors express; The hand-eye calibration homogeneous transformation matrix X is transformed by the rotation matrix R X and the translation vector t X indicates; Furthermore, the overdetermined linear equation composed of the above connected equations can be solved by the least squares method to obtain the hand-eye matrix X.

4. The weld feature recognition and path extraction method based on line laser measurement as claimed in claim 1, characterized in that: In step S2, after the hand-eye calibration experiment, the camera scans the weld to be detected, and obtains the reconstructed point cloud of the weld relative to the robot base coordinates through the robot posture, hand-eye matrix, and camera measurement point cloud; the robot posture measurement data and camera measurement data are processed, combined with the hand-eye matrix X, and the formula A is used. i XC i =p i , reconstruct the weld scanning point cloud.

5. The weld feature recognition and path extraction method based on line laser measurement as claimed in claim 1, characterized in that: In the step S3, for the reconstruction of the weld point cloud, the reconstructed point cloud needs to be preprocessed first; the uneven distribution of the weld point cloud in the horizontal and vertical directions may be caused by the uneven movement speed of the robot, and the uneven distribution of the point cloud may cause large errors in the calculation of local normal quantities and curvatures after the kd-tree nearest neighbor search in the subsequent processing; grid filtering and statistical filtering are used to obtain the downsampled point cloud; the least squares re-fitting is performed on the point cloud surface to generate a smooth surface; Use kd-tree to store point cloud data and record the neighboring points of each point in the point cloud. For the point cloud surface points and their neighboring point cloud clusters, use the eigenvalues ​​of the local point cloud cluster covariance matrix to further calculate the surface normal. The covariance matrix of a point and its nearby point cloud is: Among them, p i is the point and its surrounding points, is the centroid of the point set; Perform eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors. The eigenvector corresponding to the maximum eigenvalue is the main direction of the point cloud, indicating the main direction of the point cloud extension. The eigenvector corresponding to the minimum eigenvalue is the normal direction of the point cloud surface, indicating that the point cloud has the smallest change in this direction. The surface curvature is estimated by the eigenvalue of the covariance matrix and the surface curvature is as follows: Among them, λ0 is the minimum value among the three eigenvalues; The region growing algorithm sorts the points according to the curvature value of the point cloud; the region growth starts from the point with the smallest curvature, and the selected point is added to the set of seed points; when making the neighborhood set of the seed point, when the angle between the normal of the adjacent point and the normal of the seed point is less than the threshold, and the curvature value of the neighborhood point of the point is less than the threshold, the adjacent point is added to the set of seed points; then the current seed point is removed from the seed list, and the steps of finding the seed point set are repeated until the seed list is empty and the region growing algorithm is completed; When extracting the weld centerline, the planes on both sides of the weld are fitted as a whole, the distance from the weld point cloud to the plane is calculated, the highest point of the weld is found, and the highest point of the weld is fitted as a straight line, which is the weld centerline.

6. The weld feature recognition and path extraction method based on line laser measurement as claimed in claim 1, characterized in that: In step S4, for the weld extracted in S3, the PCA algorithm is used to calculate the main orientation of the weld to obtain the weld orientation vector; since the point cloud obtained by scanning with the line laser sensor is arranged in rows, the weld center point of each row has been extracted in S3, the farthest point on the left and right sides of the extracted weld from the center point is found as the weld edge point, and the distance between the left and right points of the center point is calculated, which is recorded as l; According to the main orientation vector of the weld and the center point of the weld, calculate the plane passing through the center point of the weld and with the main orientation vector of the weld as the normal, and record it as the center line perpendicular plane; calculate the projection length of the distance between the two center points on the main orientation vector and the projection line segment length on the center line perpendicular plane respectively; the projection line segment length on the center line perpendicular plane is the weld width.

7. A weld feature recognition and path extraction system based on line laser measurement that implements the weld feature recognition and path extraction method based on line laser measurement as claimed in any one of claims 1 to 6, characterized in that: The weld feature recognition and path extraction system based on line laser measurement includes: The calibration module is used to calibrate the hand-eye matrix of the robot using the hand-eye calibration experiment to obtain the coordinate transformation matrix of the origin of the measurement point cloud of the line laser camera relative to the origin of the robot flange; The controller module is used to synchronously record the line laser camera scanning point cloud data and the terminal coordinates and posture of the robot relative to the base coordinate system using the controller module, and reconstruct the weld scanning point cloud according to the calculated hand-eye matrix, robot motion data, and line laser scanning data. The coordinate system of the reconstructed point cloud is the robot base coordinate system; The reconstruction module is used to perform preprocessing for subsequent point cloud local feature calculation based on the reconstructed weld point cloud, extract the weld position based on the calculated point cloud local geometric features and use the clustering method; fit the plane where the weld is located, and obtain the weld centerline by extracting the highest point on the weld from the plane; The calculation module is used to calculate the vector of the main direction of the weld using the PCA algorithm based on the extracted weld, which is called the centerline vector, and the plane passing through the highest point of the weld and perpendicular to the centerline vector is the centerline vertical plane; for each single laser line, find the point farthest from the center point of the weld to the left and right to obtain the edge point of the weld; on the same laser line, calculate the distance between two points on the edge of the weld, calculate the projection length of the line segment length between the two points on the center line of the weld, and calculate the projection length of the line segment length on the centerline vertical plane; the projection length of the line segment length on the centerline vertical plane is considered to be the weld width, and the distance between the point on the center line of the weld and the fitting plane of the plate where the weld is located is the weld height.

8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the weld feature recognition and path extraction method based on line laser measurement as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the weld feature recognition and path extraction method based on line laser measurement as claimed in any one of claims 1 to 6.

10. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the weld feature recognition and path extraction system based on line laser measurement as described in claim 7.

Images