A weld feature recognition and path extraction method based on line laser measurement

Abstract

The present invention belongs to the technical field of three-dimensional measurement and weld processing, and discloses a weld feature recognition and path extraction method based on line laser measurement. The present invention proposes a fully automated method for weld morphology extraction, which can realize the automated measurement part before processing steps such as weld grinding, so that after the robot measures the weld position, it can realize weld morphology extraction and calculate the weld centerline position to obtain the processing path. At the same time, the hand-eye matrix can directly guide the robot to process the weld. In addition, the weld width and residual height features before and after weld processing can also be calculated and compared. After the weld width and residual height calculation are realized, further calculations can be performed on the processing parameters. The entire method provides technical support for automated weld processing.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional measurement and weld processing, and in particular relates to a weld feature recognition and path extraction method based on line laser measurement. Background Art

[0002] With the gradual expansion of intelligent manufacturing and digitalization, industrial robots are increasingly being used in engineering projects. Traditional processing projects require automated iteration. For example, weld grinding and welding tasks require robots to locate the weld during the task. Simultaneously, after the robot captures the image with a camera, weld feature calculations must be performed. To "acquire" the weld position, the robot must determine the weld's coordinates relative to the robot's base coordinate system. However, camera measurement only provides the distance of the measured object relative to the camera's measurement origin, and the resulting 3D point cloud is relative to the camera's origin. Therefore, a sufficiently accurate hand-eye calibration experiment is required to obtain a hand-eye calibration matrix, which determines the relative coordinates of the camera's measurement origin relative to the robot's flange. After obtaining the weld point cloud, the weld position and centerline must be extracted, and the weld width and reinforcement height must be calculated. After extracting the weld centerline, the weld can be processed further, such as grinding. The weld processing performance can be estimated based on the weld width and reinforcement height. Therefore, developing accurate hand-eye calibration methods and methods for weld extraction and feature calculation is a major trend in the advancement of automated and intelligent processing.

[0003] Through the above analysis, the problems and defects of the prior art are as follows:

[0004] (1) Based on the method of obtaining weld information from the two-dimensional image of the weld, since the two-dimensional image lacks information of one dimension, it is difficult to obtain the height information of the weld and the relative position of the weld with respect to the robot.

[0005] (2) In the past, methods for directly extracting welds and calculating features through point clouds mostly combined two-dimensional image features, which made it difficult to directly analyze point clouds to obtain weld features.

[0006] (3) Previous processing methods based on weld point clouds did not combine the robot posture to realize weld extraction. They could not directly obtain the position coordinates of the extracted weld relative to the robot, nor could they combine the weld surface to obtain the relative posture of the robot during processing.

[0007] The difficulty of solving the above problems and defects is:

[0008] A weld extraction method and weld feature calculation method based on the local geometric features of point clouds are proposed, which can directly obtain the three-dimensional information of the weld; the hand-eye calibration method can be used to obtain the relative position of the weld with respect to the robot, which can realize fully automatic recognition of the weld and make direct preparations for subsequent processing. Summary of the Invention

[0009] In response to the problems existing in the prior art, the present invention provides a weld feature recognition and path extraction method based on line laser measurement.

[0010] The present invention is implemented as follows: a weld feature recognition and path extraction method based on line laser measurement includes:

[0011] S1. Use the hand-eye calibration experiment to calibrate the hand-eye matrix of the robot and obtain the coordinate transformation matrix of the origin of the line laser camera's measurement point cloud relative to the origin of the robot flange.

[0012] S2. Use the controller module to synchronously record the line laser camera scanning point cloud data and the end coordinates and posture of the robot relative to the base coordinate system, and reconstruct the weld scanning point cloud based on the calculated hand-eye matrix, robot motion data, and line laser scanning data. The coordinate system of the reconstructed point cloud is the robot base coordinate system;

[0013] S3. Based on the reconstructed weld point cloud, preprocessing for subsequent point cloud local feature calculation is performed. The weld position is extracted using a clustering method based on the calculated local geometric features of the point cloud. The weld plane is fitted and the weld centerline is obtained by extracting the highest point on the weld from the plane.

[0014] S4. Based on the extracted weld, the PCA algorithm is used to calculate the vector of the main direction of the weld, which is called the centerline vector, and the plane passing through the highest point of the weld and perpendicular to the centerline vector is the centerline perpendicular plane; for each single laser line, find the point farthest from the center point of the weld to the left and right to obtain the weld edge point; on the same laser line, calculate the distance between the two points on the weld edge, calculate the projection length of the line segment between the two points on the weld centerline, and calculate the projection length of the line segment length on the centerline perpendicular plane; the projection length of the line segment length on the centerline perpendicular plane is considered to be the weld width, and the distance between the point on the weld centerline and the fitting plane of the plate where the weld is located is the weld height.

[0015] Furthermore, in step S1, by formula A i Xc i =p i Establish an overdetermined linear equation and solve the hand-eye matrix X, where A i is the robot pose, c i is the point cloud position captured by the camera, p i is the position of the same point in the point cloud in the robot's base system;

[0016] In the specific experiment, the standard sphere was fixed on a turntable, and the robot measured the cross section of the standard sphere in multiple postures. Based on the cross section of the standard sphere, the center of the circular cross section was fitted using the least squares method, and the coordinates of the sphere center relative to the camera were calculated based on the nominal diameter of the standard sphere.

[0017]

[0018] x circle 、z circle is the point cloud position on the circular cross section obtained by measurement, where x ball 、y ball 、z ball is the calculated coordinate of the sphere center relative to the camera, R is the standard radius of the standard sphere, and r is the cross-sectional radius calculated from the circular cross section.

[0019] Furthermore, according to the robot pose homogeneous matrix A i , unknown hand-eye matrix X, the center position of the ball in the camera coordinate system The center position of the ball in the robot base coordinate system Available The method for obtaining the hand-eye calibration equation is based on the principle that the center position of the standard ball remains unchanged. According to the measurement data, the following equation can be obtained:

[0020]

[0021] Among them, at the i-th measurement, The robot's i-th posture homogeneous transformation matrix A i Using the rotation matrix and translation vectors express; The hand-eye calibration homogeneous transformation matrix X is converted to the rotation matrix R X and the translation vector t X indicates; and

[0022] Furthermore, the overdetermined linear equation composed of the above connected equations can be solved by the least squares method to obtain the hand-eye matrix X.

[0023] Furthermore, in step S2, after the hand-eye calibration experiment, the camera scans the weld to be detected, and obtains the weld reconstructed point cloud relative to the robot base coordinates through the robot posture, hand-eye matrix, and camera measurement point cloud; the robot posture measurement data and camera measurement data are processed, combined with the hand-eye matrix X, and the formula A is used. i Xc i =p i , reconstruct the weld scanning point cloud.

[0024] Furthermore, in step S3, the reconstructed point cloud needs to be preprocessed for the weld reconstruction. The uneven distribution of the weld point cloud in the horizontal and vertical directions may be caused by the uneven movement speed of the robot. The uneven distribution of the point cloud may lead to large errors in the calculation of local normal quantities and curvatures after using kd-tree nearest neighbor search in subsequent processing. Grid filtering and statistical filtering are used to obtain the downsampled point cloud. The point cloud surface is refitted by least squares to produce a smooth surface.

[0025] Use kd-tree to store point cloud data and record the neighboring points of each point in the point cloud. For the surface points of the point cloud and their neighboring point cloud clusters, the eigenvalues ​​of the covariance matrix of the local point cloud cluster are used to further calculate the surface normal. The covariance matrix of a point and its nearby point cloud is:

[0026]

[0027] Among them, p i is the point and its surrounding points, is the centroid of the point set;

[0028] Perform eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors. The eigenvector corresponding to the maximum eigenvalue is the main direction of the point cloud, indicating the main direction of the point cloud extension. The eigenvector corresponding to the minimum eigenvalue is the normal direction of the point cloud surface, indicating that the point cloud has the least change in this direction. The surface curvature is estimated by the eigenvalue of the covariance matrix and the surface curvature is as follows:

[0029]

[0030] Among them, λ0 is the minimum value among the three eigenvalues;

[0031] The region growing algorithm sorts the point cloud points according to their curvature values; the region growth starts from the point with the smallest curvature, and the selected point is added to the set of seed points; when making the neighborhood set of the seed point, when the angle between the normal of the adjacent point and the normal of the seed point is less than the threshold, and the curvature value of the neighboring point of the point is less than the threshold, the adjacent point is added to the set of seed points; then the current seed point is removed from the seed list, and the steps of finding the seed point set are repeated until the seed list is empty, and the region growing algorithm is completed;

[0032] When extracting the weld centerline, the planes on both sides of the weld are fitted as a whole, the distance from the weld point cloud to the plane is calculated, the highest point of the weld is found, and the highest point of the weld is fitted as a straight line, which is the weld centerline.

[0033] Furthermore, in step S4, for the weld extracted in S3, the PCA algorithm is used to calculate the main orientation of the weld to obtain the weld orientation vector; since the point cloud obtained by the line laser sensor scanning is arranged in rows, the weld center point of each row has been extracted in S3, and the point farthest to the left and right of the center point in the extracted weld is found as the weld edge point, and the distance between the two points on the left and right of the center point is calculated, which is recorded as l;

[0034] According to the main direction vector of the weld and the center point of the weld, calculate the plane passing through the center point of the weld and with the main direction vector of the weld as the normal, and record it as the centerline perpendicular plane; calculate the projection length of the distance between the two center points on the main direction vector and the projection line segment length on the centerline perpendicular plane respectively; the projection line segment length on the centerline perpendicular plane is the weld width.

[0035] Another object of the present invention is to provide a weld feature recognition and path extraction system based on line laser measurement, comprising:

[0036] The calibration module is used to calibrate the robot's hand-eye matrix using a hand-eye calibration experiment to obtain the coordinate transformation matrix of the line laser camera's measurement point cloud origin relative to the robot's flange origin;

[0037] The controller module is used to synchronously record the line laser camera scanning point cloud data and the end coordinates and posture of the robot relative to the base coordinate system using the controller module, and reconstruct the weld scanning point cloud based on the calculated hand-eye matrix, robot motion data, and line laser scanning data. The coordinate system of the reconstructed point cloud is the robot base coordinate system;

[0038] The reconstruction module is used to perform preprocessing for the subsequent calculation of local features of the point cloud based on the reconstructed weld point cloud. The weld position is extracted using a clustering method based on the calculated local geometric features of the point cloud. The weld plane is fitted and the weld centerline is obtained by extracting the highest point on the weld from the plane.

[0039] The calculation module is used to calculate the vector of the main direction of the weld using the PCA algorithm based on the extracted weld, which is called the centerline vector, and the plane passing through the highest point of the weld and perpendicular to the centerline vector is the centerline perpendicular plane; for each single laser line, find the point farthest from the center point of the weld to the left and right to obtain the weld edge point; on the same laser line, calculate the distance between the two points on the weld edge, calculate the projection length of the line segment between the two points on the weld centerline, and calculate the projection length of the line segment length on the centerline perpendicular plane; the projection length of the line segment length on the centerline perpendicular plane is considered to be the weld width, and the distance between the point on the weld centerline and the fitting plane of the plate where the weld is located is the weld height.

[0040] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the weld feature recognition and path extraction method based on line laser measurement.

[0041] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the weld feature recognition and path extraction method based on line laser measurement.

[0042] Another object of the present invention is to provide an information data processing terminal, which is used to implement the weld feature recognition and path extraction system based on line laser measurement.

[0043] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0044] First, this invention proposes a fully automated method for weld seam topography extraction, enabling automated measurement prior to weld grinding and other processing steps. This allows a robot to measure the weld position, extract the weld topography, calculate the weld centerline position, and thus determine the processing path. Furthermore, the hand-eye matrix directly guides the robot in weld processing. Furthermore, the weld width and reinforcement characteristics before and after processing can be calculated and compared. After calculating the weld width and reinforcement, further calculations can be performed on processing parameters. This entire method provides technical support for automated weld processing.

[0045] Second, as auxiliary evidence for the inventiveness of the claims of the present invention, it is also reflected in the following important aspects:

[0046] (1) The expected benefits and commercial value of the technical solution of the present invention after transformation are:

[0047] By scanning the welds with a line laser sensor and designing a point cloud processing algorithm based on weld characteristics, combined with hand-eye calibration, weld processing can be automated, significantly reducing manual intervention and lowering labor costs. At the same time, robotic operation reduces human operational errors, improving the stability of the production line and the consistency of weld processing. In addition, by combining real-time scanning with point cloud analysis, the entire welding process can be automatically monitored, and any welds that do not meet the standards can be detected and automatically corrected, reducing rework and defect rates. Recording and storing real-time data from weld scanning and grinding processes can provide rich digital support for quality monitoring, production scheduling, and post-analysis, and further optimize welding and grinding processes. In small-batch production, automated grinding technology can quickly switch between different workpieces, improving production flexibility.

[0048] (2) The technical solution of the present invention fills the technical gap in the industry at home and abroad:

[0049] Traditional weld grinding relies on manual operation, which has low precision and insufficient stability, and is prone to uneven grinding or damage to the weld. In recent years, many scholars at home and abroad have studied automated welding and grinding technology for welds, but there are no detailed three-dimensional point cloud processing algorithms for different types of butt welds. For planar butt welds, many weld position acquisition methods based on depth cameras have been proposed. However, this method based on the combination of two-dimensional image processing and depth maps has low precision, is easily affected by ambient light, and cannot effectively process complex geometric shapes. The method of acquiring point clouds based on line laser sensors proposed in the present invention integrates multiple steps such as robot hand-eye calibration, line laser scanning, point cloud reconstruction and analysis, and machine learning, which can realize a complete automated processing closed loop. Point cloud scanning and analysis are completed automatically by the system, improving efficiency and accuracy. Combined with the weld width and excess height distributed along the length of the weld calculated from the weld point cloud data, the automated grinding system can provide real-time feedback on the status of the weld, and automatically plan the grinding path and force based on the feedback, thereby realizing dynamic adaptive control and providing more precise grinding effects.

[0050] Third, in the welding manufacturing industry, weld feature recognition and path extraction technologies still face multiple technical challenges. On the one hand, traditional weld feature recognition methods rely on manual calibration or simple two-dimensional image processing techniques, making it difficult to achieve three-dimensional reconstruction and accurate feature extraction of welds. On the other hand, existing weld path planning methods perform poorly in complex welding scenarios and cannot effectively cope with changes in weld geometry, such as dynamic adjustment of weld width, height, and direction. In addition, noise interference in point cloud data and insufficient accuracy in the multi-sensor data fusion process further limit the reliability of weld recognition and path extraction.

[0051] This paper proposes a weld feature recognition and path extraction method based on local geometric features. This method uses hand-eye calibration to achieve precise spatial mapping between a line laser camera and a robotic system, ensuring high-precision synchronization of point cloud data and robot position information. By preprocessing point cloud data and analyzing local features, clustering algorithms and principal component analysis (PCA) are employed to efficiently extract geometric features such as weld position, width, and height, significantly improving weld recognition accuracy. Furthermore, a dynamically adjusted weld path planning strategy effectively addresses the complex and variable weld geometry.

[0052] This invention has broad application in fields such as shipbuilding, automotive manufacturing, steel structure construction, and aerospace, providing reliable technical support for high-precision weld identification and welding path planning. By enabling efficient collaboration between line laser scanning and robotic systems, this invention significantly reduces the need for manual intervention in the welding process, improving the automation level and production efficiency of the welding process. Furthermore, by accurately extracting weld characteristics, this invention can significantly reduce the incidence of welding defects and improve product quality and reliability.

[0053] By incorporating advanced line laser scanning and dynamic algorithm optimization technology, this invention achieves a technological breakthrough in welding path planning and welding automation, addressing the inaccuracies and inefficiencies inherent in traditional methods. The industrial application of this technology will not only significantly enhance the intelligence of welding production lines but also provide strong technical support for the digitalization of the manufacturing industry, offering a novel solution for companies to reduce production costs and enhance their market competitiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a flow chart of a weld feature recognition and path extraction method based on line laser measurement provided by an embodiment of the present invention.

[0055] Figure 2 This is a structural block diagram of a weld feature recognition and path extraction system based on line laser measurement provided by an embodiment of the present invention.

[0056] Figure 3 This is a layout diagram of the experimental equipment provided by an embodiment of the present invention.

[0057] Figure 4 This is the original weld image provided by an embodiment of the present invention.

[0058] Figure 5 This is a weld centerline and edge extraction diagram provided by an embodiment of the present invention.

[0059] Figure 6 This is a schematic diagram of weld centerline extraction provided by an embodiment of the present invention.

[0060] Figure 7 This is a distribution diagram of the weld width along the y-axis of the weld provided by an embodiment of the present invention.

[0061] Figure 8 This is a distribution diagram of weld height along the y-axis length of the weld provided by an embodiment of the present invention.

[0062] Figure 9 This is a rendering of the edge point and center point extraction of a flat plate weld before and after grinding provided by an embodiment of the present invention.

[0063] Figure 10 This is a distribution diagram of the weld height and weld width of the ground weld along the y-axis provided by an embodiment of the present invention.

[0064] Figure 11 This is Example 2 provided by an embodiment of the present invention: a schematic diagram of angle butt weld extraction and edge point extraction. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0066] like Figure 1 As shown, a weld feature recognition and path extraction method based on line laser measurement provided by an embodiment of the present invention includes the following steps:

[0067] S1. Use the hand-eye calibration experiment to calibrate the hand-eye matrix of the robot and obtain the coordinate transformation matrix of the origin of the line laser camera's measurement point cloud relative to the origin of the robot flange.

[0068] S2. Use the controller module to synchronously record the line laser camera scanning point cloud data and the end coordinates and posture of the robot relative to the base coordinate system, and reconstruct the weld scanning point cloud based on the calculated hand-eye matrix, robot motion data, and line laser scanning data. The coordinate system of the reconstructed point cloud is the robot base coordinate system;

[0069] S3. Based on the reconstructed weld point cloud, preprocessing for subsequent point cloud local feature calculation is performed. The weld position is extracted using a clustering method based on the calculated local geometric features of the point cloud. The weld plane is fitted and the weld centerline is obtained by extracting the highest point on the weld from the plane.

[0070] S4. Based on the extracted weld, the PCA algorithm is used to calculate the vector of the main direction of the weld, which is called the centerline vector, and the plane passing through the highest point of the weld and perpendicular to the centerline vector is the centerline perpendicular plane; for each single laser line, find the point farthest from the center point of the weld to the left and right to obtain the weld edge point; on the same laser line, calculate the distance between the two points on the weld edge, calculate the projection length of the line segment between the two points on the weld centerline, and calculate the projection length of the line segment length on the centerline perpendicular plane; the projection length of the line segment length on the centerline perpendicular plane is considered to be the weld width, and the distance between the point on the weld centerline and the fitting plane of the plate where the weld is located is the weld height.

[0071] In step S1 provided in the embodiment of the present invention, by formula A i Xc i =p i Establish an overdetermined linear equation and solve the hand-eye matrix X, where A i is the robot pose, c iis the point cloud position captured by the camera, p i is the position of the same point in the point cloud in the robot's base system;

[0072] In the specific experiment, the standard sphere was fixed on a turntable, and the robot measured the cross section of the standard sphere in multiple postures. Based on the cross section of the standard sphere, the center of the circular cross section was fitted using the least squares method, and the coordinates of the sphere center relative to the camera were calculated based on the nominal diameter of the standard sphere.

[0073]

[0074] x circle 、z circle is the point cloud position on the circular cross section obtained by measurement, where x ball 、y ball 、z ball is the calculated coordinate of the sphere center relative to the camera, R is the standard radius of the standard sphere, and r is the cross-sectional radius calculated from the circular cross section.

[0075] The embodiment of the present invention provides a robot posture homogeneous matrix A i , unknown hand-eye matrix X, the center position of the ball in the camera coordinate system The center position of the ball in the robot base coordinate system Available The method for obtaining the hand-eye calibration equation is based on the principle that the center position of the standard ball remains unchanged. According to the measurement data, the following equation can be obtained:

[0076]

[0077] Among them, at the i-th measurement, The robot's i-th posture homogeneous transformation matrix A i Using the rotation matrix and translation vectors express; The hand-eye calibration homogeneous transformation matrix X is converted to the rotation matrix R X and the translation vector t X indicates; and

[0078] Furthermore, the overdetermined linear equation composed of the above connected equations can be solved by the least squares method to obtain the hand-eye matrix X.

[0079] In step S2 provided by the embodiment of the present invention, after the hand-eye calibration experiment, the camera scans the weld to be detected, and obtains the weld reconstructed point cloud relative to the robot base coordinates through the robot posture, hand-eye matrix, and camera measurement point cloud; the robot posture measurement data and camera measurement data are processed, combined with the hand-eye matrix X, and the formula A is used. i Xc i =p i, reconstruct the weld scanning point cloud.

[0080] In step S3 provided by the embodiment of the present invention, for reconstructing the weld point cloud, the reconstructed point cloud needs to be preprocessed first; the uneven distribution of the weld point cloud in the horizontal and vertical directions may be caused by the uneven movement speed of the robot, and the uneven distribution of the point cloud may cause large errors when calculating the local normal quantity and curvature after using the kd-tree nearest neighbor search in subsequent processing; grid filtering and statistical filtering are used to obtain the downsampled point cloud; the least squares re-fitting is performed on the point cloud surface to produce a smooth surface;

[0081] Use kd-tree to store point cloud data and record the neighboring points of each point in the point cloud. For the surface points of the point cloud and their neighboring point cloud clusters, the eigenvalues ​​of the covariance matrix of the local point cloud cluster are used to further calculate the surface normal. The covariance matrix of a point and its nearby point cloud is:

[0082]

[0083] Among them, p i is the point and its surrounding points, is the centroid of the point set;

[0084] Perform eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors. The eigenvector corresponding to the maximum eigenvalue is the main direction of the point cloud, indicating the main direction of the point cloud extension. The eigenvector corresponding to the minimum eigenvalue is the normal direction of the point cloud surface, indicating that the point cloud has the least change in this direction. The surface curvature is estimated by the eigenvalue of the covariance matrix and the surface curvature is as follows:

[0085]

[0086] Among them, λ0 is the minimum value among the three eigenvalues;

[0087] The region growing algorithm sorts the point cloud points according to their curvature values; the region growth starts from the point with the smallest curvature, and the selected point is added to the set of seed points; when making the neighborhood set of the seed point, when the angle between the normal of the adjacent point and the normal of the seed point is less than the threshold, and the curvature value of the neighboring point of the point is less than the threshold, the adjacent point is added to the set of seed points; then the current seed point is removed from the seed list, and the steps of finding the seed point set are repeated until the seed list is empty, and the region growing algorithm is completed;

[0088] When extracting the weld centerline, the planes on both sides of the weld are fitted as a whole, the distance from the weld point cloud to the plane is calculated, the highest point of the weld is found, and the highest point of the weld is fitted as a straight line, which is the weld centerline.

[0089] In step S4 provided by the embodiment of the present invention, the PCA algorithm is used to calculate the main orientation of the weld extracted in S3 to obtain the weld orientation vector; since the point cloud obtained by the line laser sensor scanning is arranged in rows, the weld center point of each row has been extracted in S3, and the point farthest to the left and right of the center point in the extracted weld is found as the weld edge point, and the distance between the two points on the left and right of the center point is calculated, which is recorded as l;

[0090] According to the main direction vector of the weld and the center point of the weld, calculate the plane passing through the center point of the weld and with the main direction vector of the weld as the normal, and record it as the centerline perpendicular plane; calculate the projection length of the distance between the two center points on the main direction vector and the projection line segment length on the centerline perpendicular plane respectively; the projection line segment length on the centerline perpendicular plane is the weld width.

[0091] A hand-eye calibration experiment determines the spatial relationship between the coordinates of the robot's end-of-arm and the point cloud origin of the line laser camera. The goal of this step is to obtain a hand-eye matrix, which represents the coordinate transformation of the point cloud origin measured by the line laser camera relative to the origin of the robot's flange. Through precise calculations in this calibration experiment, the laser scanning point cloud data and the robot motion data can be mapped into a unified coordinate system.

[0092] The controller module synchronizes the point cloud data captured by the line laser camera with the end position and posture data relative to the base coordinate system during robot motion. Using a calibrated hand-eye matrix, the point cloud data from the line laser scan is converted to the robot's base coordinate system and combined with the robot's motion data to complete 3D reconstruction of the weld scan point cloud. This process ensures that the reconstructed point cloud data has highly accurate coordinate information in space.

[0093] The reconstructed weld point cloud undergoes preprocessing, including denoising and smoothing, to ensure data continuity and accuracy. Based on this preprocessed point cloud data, local geometric features, such as normals and curvature, are calculated, and the weld location is extracted using a clustering algorithm. This step accurately locates the weld centerline by extracting the local highest point in the weld point cloud.

[0094] Fitting the highest point set of the weld point cloud extracts the weld plane and, from this, determines the weld centerline. The accuracy of the fitted plane determines the positional accuracy of the weld centerline. The weld centerline is the core of the welding path, and the accuracy of subsequent operations depends on the centerline extraction.

[0095] Based on the centerline and the highest point of the weld, principal component analysis (PCA) is used to calculate the primary orientations of the weld and determine the weld centerline vector. The weld edge points on either side of the weld are analyzed by establishing a plane perpendicular to the centerline vector, known as the centerline perpendicular plane. For each laser line, the points furthest from the weld center are identified as weld edge points, and their geometric relationship is calculated.

[0096] For each laser line, the weld width is defined as the projection length of the weld edge point on a plane perpendicular to the centerline; the weld height is defined as the vertical distance from the highest point of the weld to the fitting plane of the weld plane. Combining the overall point cloud data, the weld geometry is generated, providing a reliable data foundation for subsequent welding path planning. Ultimately, this allows for the precise extraction of weld characteristic parameters and path optimization.

[0097] like Figure 2 As shown, an embodiment of the present invention provides a weld feature recognition and path extraction system based on line laser measurement, including:

[0098] The calibration module is used to calibrate the robot's hand-eye matrix using a hand-eye calibration experiment to obtain the coordinate transformation matrix of the line laser camera's measurement point cloud origin relative to the robot's flange origin;

[0099] The controller module is used to synchronously record the line laser camera scanning point cloud data and the end coordinates and posture of the robot relative to the base coordinate system using the controller module, and reconstruct the weld scanning point cloud based on the calculated hand-eye matrix, robot motion data, and line laser scanning data. The coordinate system of the reconstructed point cloud is the robot base coordinate system;

[0100] The reconstruction module is used to perform preprocessing for the subsequent calculation of local features of the point cloud based on the reconstructed weld point cloud. The weld position is extracted using a clustering method based on the calculated local geometric features of the point cloud. The weld plane is fitted and the weld centerline is obtained by extracting the highest point on the weld from the plane.

[0101] The calculation module is used to calculate the vector of the main direction of the weld using the PCA algorithm based on the extracted weld, which is called the centerline vector, and the plane passing through the highest point of the weld and perpendicular to the centerline vector is the centerline perpendicular plane; for each single laser line, find the point farthest from the center point of the weld to the left and right to obtain the weld edge point; on the same laser line, calculate the distance between the two points on the weld edge, calculate the projection length of the line segment between the two points on the weld centerline, and calculate the projection length of the line segment length on the centerline perpendicular plane; the projection length of the line segment length on the centerline perpendicular plane is considered to be the weld width, and the distance between the point on the weld centerline and the fitting plane of the plate where the weld is located is the weld height.

[0102] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the weld feature recognition and path extraction method based on line laser measurement.

[0103] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the weld feature recognition and path extraction method based on line laser measurement.

[0104] Another object of the present invention is to provide an information data processing terminal, which is used to implement the weld feature recognition and path extraction system based on line laser measurement.

[0105] The present invention is specifically implemented:

[0106] like Figure 3 This example uses an ABB robot, a Beckhoff controller, a Keyence LJ-X8080 line laser camera, a Keyence camera controller, and an EL2262 terminal to build a measurement hardware platform. The Beckhoff controller automatically reads the robot's position, while the EL2262 terminal sends a trigger signal to simultaneously read the robot's position and measure point cloud data with the line laser camera.

[0107] A hand-eye calibration experiment was conducted. 20 sets of standard sphere data were collected for the standard sphere cross section to ensure that the line laser was always located on one side of the sphere center. The robot posture data and the corresponding line laser measurement point cloud data were recorded. The hand-eye calibration matrix was obtained according to the algorithm. The standard sphere cross section circle measured by the line laser camera was used to perform the least squares fitting algorithm on the cross section point cloud to obtain the cross section circle radius and the center position. The x, y, and z coordinates of the standard sphere center were obtained by combining the standard sphere radius. The x, y, and z coordinates of the standard sphere center were obtained by the following formula 1. ball 、y ball 、z ball :

[0108]

[0109] where x ball 、y ball 、z ball is the calculated coordinate of the sphere center relative to the camera, x circle 、z circle is the center position of the circle on the measured circular section, R is the standard radius of the standard sphere, and r is the cross-sectional radius calculated from the circular section.

[0110] Assume that the robot pose homogeneous matrix is ​​A i , the unknown hand-eye matrix is ​​X, the center position of the ball in the camera coordinate system is

[0111] Set to The position of the ball center in the robot base coordinate system is Available The method for obtaining the hand-eye calibration equation is based on the principle that the center position of the standard ball remains unchanged. According to the measurement data, equation 2 can be obtained:

[0112]

[0113] Among them, at the i-th measurement, The robot's i-th posture homogeneous transformation matrix A i Using the rotation matrix and translation vectors express. The hand-eye calibration homogeneous transformation matrix X is converted to the rotation matrix R X and the translation vector t X Indicates. By solving the overdetermined linear equation composed of the above connected equations using the least squares method, the desired hand-eye matrix X can be obtained.

[0114] like Figure 4 、 Figure 5 After the hand-eye calibration experiment, weld measurement is performed. To synchronize the robot's motion pose and camera measurement information, the EL2262 terminal sends a continuous trigger signal. The host computer uses code to control the camera to batch-collect data under external triggering. Simultaneously, it records the robot's motion pose data read by the Beckhoff controller. The weld to be measured is fixed in position, the weld is measured, and the weld point cloud is reconstructed based on the hand-eye matrix.

[0115] like Figure 6 、 Figure 7 A region growing algorithm is performed on the weld point cloud, and the weld position is further extracted by Gaussian curvature. When calculating the local features of the weld, such as curvature and normal vector, a kd-tree is used to search the point cloud neighbors, and the k nearest neighbor point set of each point in the point cloud is recorded. The principal component analysis (PCA) method is used to further calculate the local nearest neighbor point set point cloud cluster to calculate the point cloud surface normal vector and curvature. Record p i is the i-th point on the point cloud, is the centroid of the set of m neighboring points of the i-th point. The covariance matrix of a point and its nearby point cloud is shown in formula 3:

[0116]

[0117] Perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalues ​​and corresponding eigenvectors. The eigenvector corresponding to the maximum eigenvalue is the main direction of the point cloud, indicating the main direction of the point cloud extension. The eigenvector corresponding to the minimum eigenvalue is the normal direction of the point cloud surface, indicating that the point cloud has the least change in this direction. Let λ0 be the minimum of the three eigenvalues, and λ1 and λ2 be the other two eigenvalues. Equation 4 estimates the i-th point p based on the eigenvalues i The surface curvature cur at:

[0118]

[0119] like Figure 8 、 Figure 9 When implementing the region growing algorithm, the point cloud is sorted according to its curvature value, so that the region growing algorithm starts from the point with the smallest curvature. The selected point is added to the set of seed points. When creating the neighborhood set of seed points, when the angle between the normal of the adjacent point and the normal of the seed, and the curvature value of the neighborhood point are less than the threshold, the adjacent point is added to the set of seed points. The next step is to remove the current seed point from the seed list. The two steps of seed point expansion and current seed point removal are repeated until the seed list is empty and the region growing algorithm is completed. After removing the plane according to the region growing algorithm, the PCL point cloud library is used to calculate the Gaussian curvature of the point cloud, and a threshold is set to extract the complete weld part.

[0120] The principal component analysis (PCA) method is used to solve the main orientation vector of the weld seam as a whole, which is recorded as Perform a plane fit on the plane separated previously and calculate the plane normal. Calculate the distance from the weld point to the plane, find the highest point on the weld, and obtain the weld reinforcement along the weld direction. Based on the weld's primary orientation and highest point, estimate the weld centerline. Since the line laser scanning point cloud is an ordered point cloud sorted by row, search for the farthest points on either side of the weld center point and identify them as weld edge points. Calculate the distance between these two points, denoted as l.

[0121] Based on the weld's principal orientation vector and the weld's center point, calculate the plane passing through the weld's center point and with the principal orientation vector as its normal. This plane is referred to as the centerline perpendicular plane. Calculate the projection length of the distance l between the two points onto the principal orientation vector and record it as l1. Also, calculate the length of the line segment projected onto the centerline perpendicular plane between the two points and record it as l2. Equation 5 gives the modulus of l1, while Equation 6 gives the modulus of l2. The modulus of l2 is the weld width.

[0122]

[0123]

[0124] The present invention relates to the field of scanning welds using a robot-held line laser camera, reconstructing weld point clouds, and developing point cloud processing algorithms. The related technologies can be applied across multiple technical fields. In the field of automated welding and weld quality inspection, the entire inventive process can be used for automated welding quality control, particularly for the precise inspection and evaluation of complex weld joints, such as planar butt joints and corner joints. During welding process development and optimization, welding parameters can be optimized by reconstructing and analyzing different weld shapes, thereby improving welding quality and production efficiency. Furthermore, in many complex engineering projects, part design files may be lost or incomplete. Scanning using a robot-held line laser camera can accurately capture the workpiece's three-dimensional geometric data and generate a complete three-dimensional model using point cloud reconstruction technology for subsequent design and analysis. In manufacturing applications, welding, polishing, and part assembly of automotive and aerospace structures are key processes. This system, which integrates a robot, a line laser camera, point cloud processing software, and intelligent algorithms, can automatically inspect weld joint quality, including weld height, width, and shape, to ensure weld quality.

[0125] The present invention provides two local weld processing effect diagrams for flat plate welds and corner butt welds, and extracts the weld edge points and center points for flat plate welds, providing effect diagrams of the changes in the weld width and the excess height along the weld length. In the effect diagram of the weld edge point and center point extraction, a schematic diagram of calculating the weld width from the distance between the weld edge points is given. Before and after the plane weld grinding experiment, it can be observed that the weld height is reduced and the weld width remains basically unchanged. When processing the complex butt weld in the example diagram, the weld and the corners of the two edge plates are segmented by clustering to obtain the weld position, and the weld extraction edge points are displayed, indicating the conversion of the distance between the two edge points of the weld to the weld width.

[0126] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art will appreciate that the above-mentioned devices and methods can be implemented using computer-executable instructions and / or contained in processor control code, for example, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, or can be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0127] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A weld feature recognition and path extraction method based on line laser measurement, characterized in that: The following steps are involved: S1. Use the hand-eye calibration experiment to calibrate the hand-eye matrix of the robot and obtain the coordinate transformation matrix of the origin of the line laser camera's measurement point cloud relative to the origin of the robot flange. S2. Use the controller module to synchronously record the line laser camera scanning point cloud data and the end coordinates and posture of the robot relative to the base coordinate system, and reconstruct the weld scanning point cloud based on the calculated hand-eye matrix, robot motion data, and line laser scanning data. The coordinate system of the reconstructed point cloud is the robot base coordinate system; S3. Based on the reconstructed weld point cloud, preprocessing is performed for subsequent point cloud local feature calculation. The weld position is extracted using a clustering method based on the calculated local geometric features of the point cloud. The weld plane is fitted and the weld centerline is obtained by extracting the highest point on the weld from the plane. S4. Based on the extracted weld, use the PCA algorithm to calculate the vector of the main direction of the weld, which is called the centerline vector, and take the plane passing through the highest point of the weld and perpendicular to the centerline vector as the centerline perpendicular plane; for each single laser line, find the point farthest from the center point of the weld to the left and right to obtain the weld edge point; on the same laser line, calculate the distance between two points on the weld edge, calculate the length of the line segment between the two points projected on the weld centerline, and calculate the length of the line segment projected on the centerline perpendicular plane; consider the length of the line segment projected on the centerline perpendicular plane to be the weld width, and the distance between the point on the weld centerline and the fitting plane of the plate where the weld is located to be the weld height; In step S1, by formula A i Xc i =p i Establish an overdetermined linear equation and solve the hand-eye matrix X, where A i is the robot pose, c i is the point cloud position captured by the camera, p i is the position of the same point in the point cloud in the robot's base system; In the specific experiment, the standard sphere was fixed on a turntable, and the robot measured the cross section of the standard sphere in multiple postures. Based on the cross section of the standard sphere, the center of the circular cross section was fitted using the least squares method, and the coordinates of the sphere center relative to the camera were calculated based on the nominal diameter of the standard sphere. x circle 、z circle is the point cloud position on the circular cross section obtained by measurement, where x ball 、y ball 、z ball is the calculated coordinate of the sphere center relative to the camera, R is the standard radius of the standard sphere, and r is the cross-sectional radius calculated from the circular cross section.

2. The weld feature recognition and path extraction method based on line laser measurement according to claim 1, characterized in that: According to the robot pose homogeneous matrix A i , unknown hand-eye matrix X, the center position of the ball in the camera coordinate system The center position of the ball in the robot base coordinate system Available According to the principle that the center position of the standard ball remains unchanged, based on the measurement data, we can get the equation: Among them, at the i-th measurement, The robot's i-th posture homogeneous transformation matrix A i Using the rotation matrix and translation vectors express; The hand-eye calibration homogeneous transformation matrix X is converted to the rotation matrix R X and the translation vector t X indicates; and Furthermore, the overdetermined linear equation composed of the above connected equations can be solved by the least squares method to obtain the hand-eye matrix X.

3. The weld feature recognition and path extraction method based on line laser measurement according to claim 1, characterized in that: In step S2, after the hand-eye calibration experiment, the camera scans the weld to be detected, and obtains the weld reconstructed point cloud relative to the robot base coordinates through the robot posture, hand-eye matrix, and camera measurement point cloud; the robot posture measurement data and camera measurement data are processed, combined with the hand-eye matrix X, and the formula A is used. i Xc i =p i , reconstruct the weld scanning point cloud.

4. The weld feature recognition and path extraction method based on line laser measurement according to claim 1, characterized in that: In step S3, the reconstructed point cloud needs to be preprocessed in order to reconstruct the weld point cloud. The uneven distribution of the weld point cloud in the horizontal and vertical directions may be caused by the uneven movement speed of the robot. The uneven distribution of the point cloud may lead to large errors in the calculation of local normal quantities and curvature after using kd-tree nearest neighbor search in subsequent processing. Grid filtering and statistical filtering are used to obtain the downsampled point cloud. The surface of the point cloud is refitted by least squares to produce a smooth surface. Use kd-tree to store point cloud data and record the neighboring points of each point in the point cloud. For the surface points of the point cloud and their neighboring point cloud clusters, the eigenvalues ​​of the covariance matrix of the local point cloud cluster are used to further calculate the surface normal. The covariance matrix of a point and its nearby point cloud is: Among them, p i is the point and its surrounding points, is the centroid of the point set; Perform eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors. The eigenvector corresponding to the maximum eigenvalue is the main direction of the point cloud, indicating the main direction of the point cloud extension. The eigenvector corresponding to the minimum eigenvalue is the normal direction of the point cloud surface, indicating that the point cloud has the least change in this direction. The surface curvature is estimated by the eigenvalue of the covariance matrix and the surface curvature is as follows: Among them, λ0 is the minimum value among the three eigenvalues; The region growing algorithm sorts the point cloud points according to their curvature values; the region growth starts from the point with the smallest curvature, and the selected point is added to the set of seed points; when making the neighborhood set of the seed point, when the angle between the normal of the adjacent point and the normal of the seed point is less than the threshold, and the curvature value of the neighboring point of the point is less than the threshold, the adjacent point is added to the set of seed points; then the current seed point is removed from the seed list, and the steps of finding the seed point set are repeated until the seed list is empty, and the region growing algorithm is completed; When extracting the weld centerline, the planes on both sides of the weld are fitted as a whole, the distance from the weld point cloud to the plane is calculated, the highest point of the weld is found, and the highest point of the weld is fitted as a straight line, which is the weld centerline.

5. The weld feature recognition and path extraction method based on line laser measurement according to claim 1, characterized in that: In step S4, for the weld extracted in S3, the PCA algorithm is used to calculate the main orientation of the weld to obtain the weld orientation vector; since the point cloud obtained by the line laser sensor scanning is arranged in rows, the weld center point of each row has been extracted in S3, and the point farthest to the left and right of the center point in the extracted weld is found as the weld edge point, and the distance between the two points on the left and right of the center point is calculated, which is recorded as l; According to the main direction vector of the weld and the center point of the weld, calculate the plane passing through the center point of the weld and with the main direction vector of the weld as the normal, and record it as the centerline perpendicular plane; calculate the projection length of the distance between the two center points on the main direction vector and the projection line segment length on the centerline perpendicular plane respectively; the projection line segment length on the centerline perpendicular plane is the weld width.

6. A weld feature recognition and path extraction system based on line laser measurement that implements the weld feature recognition and path extraction method based on line laser measurement according to any one of claims 1 to 5, characterized in that: The weld feature recognition and path extraction system based on line laser measurement includes: The calibration module is used to calibrate the robot's hand-eye matrix using a hand-eye calibration experiment to obtain the coordinate transformation matrix of the line laser camera's measurement point cloud origin relative to the robot's flange origin; The controller module is used to synchronously record the line laser camera scanning point cloud data and the end coordinates and posture of the robot relative to the base coordinate system using the controller module, and reconstruct the weld scanning point cloud based on the calculated hand-eye matrix, robot motion data, and line laser scanning data. The coordinate system of the reconstructed point cloud is the robot base coordinate system; The reconstruction module is used to perform preprocessing for the subsequent calculation of local features of the point cloud based on the reconstructed weld point cloud. The weld position is extracted using a clustering method based on the calculated local geometric features of the point cloud. The weld plane is fitted and the weld centerline is obtained by extracting the highest point on the weld from the plane. The calculation module is used to calculate the vector of the main direction of the weld using the PCA algorithm based on the extracted weld, which is called the centerline vector, and the plane passing through the highest point of the weld and perpendicular to the centerline vector is the centerline perpendicular plane; for each single laser line, find the point farthest from the center point of the weld to the left and right to obtain the weld edge point; on the same laser line, calculate the distance between the two points on the weld edge, calculate the projection length of the line segment between the two points on the weld centerline, and calculate the projection length of the line segment length on the centerline perpendicular plane; the projection length of the line segment length on the centerline perpendicular plane is considered to be the weld width, and the distance between the point on the weld centerline and the fitting plane of the plate where the weld is located is the weld height.

7. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the weld feature recognition and path extraction method based on line laser measurement as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the weld feature recognition and path extraction method based on line laser measurement according to any one of claims 1 to 5.

9. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the weld feature recognition and path extraction system based on line laser measurement as described in claim 6.

Images