A real-time weld tracking method and system based on 3D point cloud

Abstract

The present invention belongs to the field of weld tracking technology, and discloses a real-time weld tracking method and system based on 3D point cloud. The method makes a prediction and replacement for abnormal points according to the previous weld points, which can improve the accuracy of weld tracking. Previous weld tracking often ignores the problem of determining the starting point and end point of welding. When invalid points appear in real-time welding, the problem of misjudgment of the weld end point will occur. For the welding starting point, previous weld tracking ignores the problem of the posture transformation of the robot from the starting position to the starting point. The present invention processes both the starting point and the end point, and uses improved B-spline fitting to eliminate the problem of uneven endpoint connection during real-time path fitting, making the entire weld tracking process more automated, reducing many places that require manual teaching, and proposing a solution to the problem of misjudgment of the weld end point.

Description

Technical Field

[0001] The present invention belongs to but is not limited to the field of weld tracking technology, and in particular relates to a real-time weld tracking method and system based on 3D point cloud. Background Art

[0002] As an essential process in the manufacturing industry, welding plays a key role in improving product quality, reducing production costs, and increasing production efficiency. With the continuous advancement of industrial informatization and intelligentization, welding technology is also gradually evolving towards intelligentization and automation. Welding robots, due to their high precision, stability, and efficiency, are gradually becoming a mainstream alternative to manual welding. To adapt to complex environmental changes during the welding process, such as assembly processing errors and welding deformation, and to further improve welding quality, intelligent welding robot systems have become a hot topic of research.

[0003] Due to their non-contact, high-speed, and high-precision advantages, visual measurement sensors are widely used in weld seam tracking systems. Currently, welding robots widely use two-dimensional visual sensors for weld seam identification and tracking. While they perform well in simple weld seam tracking tasks, their limitations are becoming increasingly apparent when handling complex three-dimensional weld seam trajectories. Two-dimensional sensors struggle to provide sufficient depth information and spatial resolution, which affects the accuracy and stability of weld seam identification and tracking. Line structured light cameras project light onto the welded workpiece and capture the deformed light image using the camera to reconstruct three-dimensional point cloud data of the welding environment. They not only provide accurate three-dimensional position and shape information of the weld seam, but also exhibit strong anti-interference capabilities and adaptability. Therefore, line structured light has attracted widespread attention in the field of weld seam tracking.

[0004] Real-time and stable tracking control is the key to efficient and accurate weld tracking. Existing weld tracking methods often scan the workpiece before welding to obtain and process the weld trajectory. Due to the deformation factors that occur during welding, the actual weld is often different from the weld before welding, resulting in errors in weld tracking, affecting the weld forming quality, and the process is complicated. Moreover, since the weld feature points are obtained in real time, the path fitting is often performed in segments, and the fitting curve connections at the segment endpoints are often not smooth enough, which causes the robot speed to fluctuate. Due to factors such as arc light and sensor stability, the feature points obtained often have abnormal points, resulting in inaccurate fitting paths and errors in autonomously determining the start and end points of the weld. Therefore, this field needs a comprehensive and accurate weld tracking method and system.

[0005] Through the above analysis, the problems and defects of the existing technology are as follows:

[0006] Existing weld seam tracking methods often scan and process the workpiece before welding to obtain the weld seam trajectory. Due to deformation factors during welding, the actual weld seam often differs from the pre-weld weld seam, resulting in weld seam tracking errors, affecting weld quality and complicating the process. Moreover, because weld feature points are acquired in real time, path fitting is often performed in segments. The fitting curve connections at the segment endpoints are often not smooth, which can cause fluctuations in robot speed. Summary of the Invention

[0007] In view of the problems existing in the prior art, the present invention provides a real-time weld seam tracking method based on 3D point cloud.

[0008] The technical solution of the present invention is as follows: a real-time weld seam tracking method based on 3D point cloud, the specific steps of which include:

[0009] S1. Select points on the weld to be welded, obtain point cloud data of the points through line structured light, identify weld feature points, and complete the coordinate conversion between the weld feature points and the robot welding end through hand-eye calibration;

[0010] S2. Teach an initial path for seam tracking. When the robot recognizes a weld feature point along the path, this point is defined as the weld starting point. The robot moves to the weld starting point along the welding speed and waits for the welding instruction before starting seam tracking.

[0011] S3. Due to the line structured light in advance, when the end of the robot reaches the starting point of the weld, the weld feature points in the weld feature point queue are uniformly discretized by improved B-spline fitting to generate smooth and accurate discrete points as the robot's motion path.

[0012] S4. Identification of the weld end point. In the weld feature point queue, considering the influence of recognition error factors, the invalid points and end points that appear in the welding are identified to complete the entire welding task.

[0013] Furthermore, the method for identifying weld feature points using the line structured light camera is as follows:

[0014] First, the point cloud is preprocessed by removing some weld points outside the weld through straight-through filtering to make the point cloud data more concentrated in the weld area. Then, the point cloud density is reduced through voxel grid filtering. The weld surface is classified through DBSCAN. For weld feature point identification, different solutions are adopted for different welds, as shown below:

[0015] For butt or V-groove welds, the classified weld surfaces are identified using the differential method, and the center point of the weld boundary point is used as the weld feature point;

[0016] For lap welds, the classified weld surfaces are used to identify the welds using the differential method, and the higher weld surface boundary is used as the weld feature point;

[0017] For fillet welds, a straight line is fitted to the classified weld surface, and the intersection of the two straight lines is the weld feature point.

[0018] Furthermore, the specific method of step S1 is:

[0019] The 10-point method is used for hand-eye calibration. First, the position of the robot end effector and the position of the identified feature points are collected at different positions and postures. At each position and posture, the robot end effector pose matrix A can be obtained. i and the camera pose matrix B i , we can construct the equation:

[0020] A i X=XB i

[0021]

[0022]

[0023] Where X is a fixed transformation matrix, R X is the rotation matrix, and The translation vectors of the robot end effector and the camera at the ith position and posture are converted into vector form, t X is the translation vector; R X SVD can be used to solve, t X Use the least squares method to solve.

[0024] Further, the step S2 method:

[0025] Manually teach an initial path and set the required welding parameters. It is necessary to ensure that the structured light can recognize the weld. When identifying the weld feature points, according to statistical principles, when two or more feature points appear consecutively, the first feature point is used as the starting point of the weld.

[0026] Further, the step S3 method:

[0027] The feature points in the weld feature point queue are sequentially fitted with an improved B-spline curve, and the fitting curve is uniformly discretized. The path point information is fed back to the robot so that it can perform the welding task according to the fitting path.

[0028] The processing of abnormal points is as follows: set a sliding window with a size of N. The value of N needs to take into account the welding speed, tracking accuracy and real-time performance; calculate the difference between two adjacent feature points in the Y and Z directions, recorded as Y i and Z i, the difference between the point to be determined and its previous feature point is recorded as Y new and Z new , the Z-score method is used to determine whether the difference is an outlier, and the mean and variance of the difference are calculated:

[0029]

[0030]

[0031]

[0032]

[0033] where μ y 、μ z is the mean difference between the Y and Z directions, σ y , σ Z is the variance of the difference in the Y and Z directions.

[0034] Then calculate the Z-score of the difference between the discriminant points:

[0035]

[0036]

[0037] When | Z new,y |>T or |Z new,z When |>T, the discriminant point can be identified as an abnormal point, where T is the discriminant value set according to the actual situation.

[0038] Further, the step S4 method:

[0039] A threshold range is set according to the taught weld end point information. When the identified weld feature point reaches this threshold range, when two or more invalid points appear consecutively, the last point is determined to be the weld end point.

[0040] Another object of the present invention is to provide a real-time weld tracking system based on 3D point cloud, comprising:

[0041] The recognition module is used to select points on the weld to be welded, obtain point cloud data of the points through line structured light, identify weld feature points, and complete the coordinate conversion between the weld feature points and the robot welding end through hand-eye calibration;

[0042] The tracking module is used to teach an initial path for seam tracking. When the robot identifies a weld feature point along the path, this point is defined as the weld starting point. The robot moves to the weld starting point along the welding speed and waits for the welding instruction before starting seam tracking.

[0043] The fitting module is used to perform improved B-spline fitting on the weld feature points in the weld feature point queue when the robot terminal reaches the weld starting point due to the line structured light advance, perform uniform discretization, and generate smooth and equidistant discrete points as the robot's motion path;

[0044] The discrimination module is used to discriminate the end point of the weld. In the weld feature point queue, it takes into account the influence of recognition error factors and discriminates the invalid points and end points that appear in the welding to complete the entire welding task.

[0045] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the real-time weld tracking method based on 3D point cloud.

[0046] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the real-time weld tracking method based on 3D point cloud.

[0047] Another object of the present invention is to provide an information data processing terminal, which is used to implement the real-time weld tracking system based on 3D point cloud.

[0048] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are analyzed from the following aspects:

[0049] First, since the entire weld path planning is generated in real time during welding, the steps of scanning and processing the point cloud before welding to obtain the welding path are eliminated. In addition, the process parameters can be adaptively adjusted according to the weld state to account for changes in weld caused by deformation during welding. This can greatly save automated welding time and ensure that the weld has a higher forming quality. In the past, the treatment of abnormal weld points was often done by directly removing them. In the present invention, a prediction is made for the abnormal points based on the previous weld points to improve the accuracy of weld tracking. Previous weld tracking often ignored the problem of determining the starting and ending points of welding. When invalid points appear in real-time welding, the problem of misjudgment of the weld end point will occur. For the welding starting point, previous weld tracking ignored the problem of the robot's posture transformation from the starting position to the starting point. The present invention performs processing at both the starting point and the end point, making the entire weld tracking process more automated, reducing many places that require manual teaching, and proposing a solution to the problem of misjudgment of the weld end point.

[0050] The technical problem to be solved by the present invention is mainly to solve the problem of weld tracking errors caused by arc light, welding deformation and sensor stability. The present invention provides a real-time weld tracking method based on 3D point cloud, which can achieve high-precision weld tracking for curved welds. The present invention only needs to manually teach an initial trajectory, and can automatically reach the starting point and start weld tracking, eliminating the pre-weld scanning process and having high recognition accuracy.

[0051] Second, the main parameters, algorithms, and mathematical models of this invention solve several key issues in the existing technology. First, during the hand-eye calibration process, traditional methods are often limited by calibration accuracy and stability. This invention significantly improves calibration accuracy and stability by utilizing a 10-point method combined with singular value decomposition (SVD) and least squares method. This method effectively solves the problem of precise positioning between the robot end effector and the camera, laying a solid foundation for subsequent welding tasks.

[0052] Secondly, when it comes to identifying the weld starting point, traditional manual teaching methods are prone to inaccurate recognition and low efficiency. This invention manually teaches the initial path and, using statistical principles, identifies the weld starting point as the first characteristic point when two or more consecutive characteristic points appear. This method not only improves the accuracy of weld starting point identification but also significantly enhances welding efficiency, ensuring consistency and reliability of the welding process.

[0053] Furthermore, the present invention has achieved significant progress in weld path fitting and outlier processing. By fitting weld feature points using an improved B-spline curve and processing outliers using a sliding window and Z-score method, the smoothness and accuracy of the weld path are effectively improved. Traditional methods often suffer from uneven weld paths and inaccurate outlier identification. However, the present invention successfully addresses these issues through sophisticated mathematical models and algorithmic design, ensuring high weld path quality.

[0054] Finally, to address the issue of identifying invalid points and weld endpoints during welding, the present invention proposes a threshold range-based discrimination method. When the identified weld feature points fall within the set threshold range and two or more invalid points appear consecutively, the weld endpoint is determined. This method effectively addresses the issue of invalid feature point identification during welding, ensuring the continuity and accuracy of welding tasks. Through these improvements, the present invention not only addresses the challenges of existing technologies but also achieves significant technological advancements, providing strong support for efficient and precise welding operations.

[0055] Second, the technical solution of the present invention fills the technical gaps in the industry at home and abroad: most of the existing weld tracking technologies focus on pre-scanning before welding to obtain weld information and then perform trajectory planning. It is often difficult to predict problems such as weld deformation during welding, which causes welding defects such as weld deviation and weld leakage. This technology is based on real-time 3D point cloud processing of welds, which can obtain the morphological changes of welds during welding in real time and plan the welding path in real time. Compared with the past, it can greatly reduce the welding defects caused by deformation. Since real-time trajectory planning is real-time, its trajectory is often planned in sections, so it is often discontinuous at the endpoints of each trajectory. The improved B-spline trajectory adopted in this solution ensures the continuity of each real-time trajectory planning endpoint while maintaining high precision and smoothness of the entire trajectory.

[0056] The technical solution of this invention solves a long-cherished but unsuccessful technical problem: the traditional robotic welding process requires long manual teaching times and high application costs. Acquiring three-dimensional information about welds based on 3D point clouds enables high-precision weld detection and tracking, eliminating manual teaching time and significantly saving costs. Manual teaching often results in large errors, particularly for narrow welds. Furthermore, changes in weld position caused by deformation during welding are often difficult to adjust in real time, impacting weld quality. This real-time tracking method, however, involves planning the welding path in real time, enabling real-time sensing and adjustment of weld changes.

[0057] Third. Technical problems solved by the present invention:

[0058] 1) Inaccurate weld identification:

[0059] In the existing technology, the weld recognition accuracy is low, especially in complex weld shapes and welding environments, it is difficult to identify weld feature points, resulting in unstable welding quality.

[0060] The present invention classifies and identifies different types of welds (butt joints, V-grooves, overlap joints, and fillet joints) by combining a line structured light camera with the DBSCAN algorithm and the differential method, thereby improving the recognition accuracy of weld feature points.

[0061] 2) Path generation is not smooth:

[0062] Traditional weld seam tracking path generation methods are difficult to ensure the smoothness of the path, resulting in unstable movement during robot welding and affecting welding quality.

[0063] The present invention generates a smooth and uniform welding path by improving the B-spline curve fitting and uniform discretization method, thereby ensuring stable movement of the robot during welding.

[0064] 3) Weld seam tracking is not real-time:

[0065] Existing weld seam tracking methods have high computational complexity and poor real-time performance, making it difficult to accurately track weld seams during dynamic welding processes.

[0066] The present invention processes weld feature points in real time through a sliding window and a Z-score method, eliminates abnormal points, and ensures the real-time and accuracy of weld tracking.

[0067] ###Significant technological advancements achieved:

[0068] 1) Improved weld recognition accuracy:

[0069] The line structured light camera is combined with the DBSCAN algorithm and the differential method to classify and identify different types of welds, which greatly improves the recognition accuracy of weld feature points and ensures the accuracy of the welding path.

[0070] 2) The welding path is smooth and stable:

[0071] By improving the B-spline curve fitting and uniform discretization method, a smooth and uniform welding path is generated to ensure the smooth movement of the robot during the welding process and improve the welding quality.

[0072] 3) Real-time weld tracking:

[0073] The sliding window and Z-score method are used to process weld feature points in real time and eliminate abnormal points to ensure the real-time and accuracy of weld tracking, so that the robot can accurately track the weld during the dynamic welding process.

[0074] 4) Strong adaptability to multiple types of welds:

[0075] The method of the present invention is applicable to various types of welds (butt joints, V-grooves, lap joints, and fillet joints), has strong adaptability, and meets the needs of different welding scenarios.

[0076] Through the above embodiments, the real-time weld seam tracking method based on 3D point cloud of the present invention significantly improves the welding accuracy and efficiency in industrial automatic welding and shipbuilding, solves the problems of inaccurate weld seam identification, uneven path generation, and non-real-time weld seam tracking in the prior art, and provides a reliable and efficient solution for related industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 This is a flow chart of a real-time weld tracking method based on 3D point cloud provided by an embodiment of the present invention.

[0078] Figure 2 This is a structural block diagram of a real-time weld tracking system based on 3D point cloud provided by an embodiment of the present invention.

[0079] Figure 3This is a hardware structure diagram of the weld tracking system provided by an embodiment of the present invention.

[0080] Figure 4 This is a detailed flow chart of a real-time weld tracking method based on 3D point cloud provided by an embodiment of the present invention.

[0081] Figure 5 This is a diagram of arc weld feature points obtained by simulating a structured light camera in an example of the present invention.

[0082] Figure 6 This is a fitting curve error diagram of the improved B-spline simulated by the example of the present invention.

[0083] Figure 7 This is a path fitting error diagram of an example simulation of the present invention under a random error of ±0.2 mm.

[0084] Figure 8 This is a welding speed error diagram of an example simulation of the present invention under a random error of ±0.2mm.

[0085] Figure 9 The improved B-spline curve fitting method of the present invention is DETAILED DESCRIPTION

[0086] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0087] Example 1: Industrial automatic welding

[0088] Industrial automatic welding requires a high-precision, high-stability weld seam tracking system suitable for complex weld shapes and dynamic welding environments.

[0089] 1) Weld identification:

[0090] The weld feature points are identified by a line structured light camera, and the DBSCAN algorithm and differential method are used for classification and identification to improve recognition accuracy.

[0091] 2) Path generation:

[0092] Improved B-spline curve is used to fit weld feature points to generate a smooth and uniform welding path, ensuring the stability of the welding process.

[0093] 3) Real-time tracking:

[0094] The sliding window and Z-score method are used to process feature points and eliminate abnormal points to ensure the real-time and accuracy of weld tracking.

[0095] Effect:

[0096] - Improve welding quality and efficiency and reduce welding defects.

[0097] -Achieve high-precision automatic welding of complex weld shapes.

[0098] Example 2: Shipbuilding

[0099] During the shipbuilding process, the welding tasks are heavy and the weld shapes are complex, which requires a high-precision weld tracking system.

[0100] 1) Weld identification:

[0101] Use a line structured light camera combined with the DBSCAN algorithm to identify the characteristic points of ship welds and improve recognition accuracy.

[0102] 2) Path generation:

[0103] The improved B-spline curve is used to fit the weld feature points to generate a smooth and uniform welding path, ensuring the stability of the welding process.

[0104] 3) Real-time tracking:

[0105] The sliding window and Z-score method are used to process feature points and eliminate abnormal points to ensure the real-time and accuracy of weld tracking.

[0106] Effect:

[0107] - Improve ship welding quality and reduce welding defects.

[0108] -Achieve high-precision automatic welding and improve production efficiency.

[0109] like Figure 1 As shown, a real-time weld seam tracking method based on 3D point cloud provided by an embodiment of the present invention includes the following steps:

[0110] S1. Select points on the weld to be welded, obtain point cloud data of the points through line structured light, identify weld feature points, and complete the coordinate conversion between the weld feature points and the robot welding end through hand-eye calibration.

[0111] S2. Teach an initial path for weld tracking. When the robot recognizes a weld feature point along the path, this point is defined as the weld starting point. It moves to the weld starting point along the welding speed and waits for welding instructions before starting weld tracking.

[0112] S3. Due to the line structured light in advance, when the end of the robot reaches the starting point of the weld, the weld feature points in the weld feature point queue are uniformly discretized by improved B-spline fitting to generate smooth and equidistant discrete points as the robot's motion path.

[0113] S4. Determination of the weld end point. In the weld feature point queue, considering the influence of factors such as recognition error, the invalid points and end points that appear in the welding are determined to complete the entire welding task.

[0114] Prioritizing the use of line structured light cameras to identify weld feature points, the specific principle is as follows: first, pre-process the point cloud by removing some weld outside points through straight-through filtering to make the point cloud data more concentrated in the weld area, then reduce the point cloud density through voxel grid filtering, and classify the weld surface through DBSCAN. For weld feature point identification, different solutions are adopted for different welds, as shown below:

[0115] For butt or V-groove welds, the classified weld surfaces are identified using the differential method, and the center points of the weld boundary points are used as weld feature points.

[0116] For lap welds, the classified weld surfaces are identified using the differential method, and the higher weld surface boundary is used as the weld feature point.

[0117] For fillet welds, a straight line is fitted to the classified weld surface, and the intersection of the two straight lines is the weld feature point.

[0118] First, the implementation principle of step S1 is as follows: use the 10-point method to perform hand-eye calibration. First, the position of the robot end effector and the position of the identified feature points are collected at different positions and postures. At each position and posture, we can obtain the robot end effector pose matrix A. i and the camera pose matrix B i , we can construct the equation:

[0119] A i X=XB i

[0120]

[0121]

[0122] Where X is a fixed transformation matrix, R X is the rotation matrix, and The translation vectors of the robot end effector and the camera at the ith position and posture are converted into vector form, t X is the translation vector. X SVD can be used to solve, t X Use the least squares method to solve.

[0123] Preferably, the implementation principle of step S2 is as follows: manually teach an initial path and set the required welding parameters, etc., wherein it is necessary to ensure that the structured light can recognize the weld. When the weld feature points are identified, according to statistical principles, when two or more feature points appear consecutively, the first feature point is used as the starting point of the weld.

[0124] Prioritize, the implementation principle of step S3 is as follows: first, the starting point and the end point of the weld are judged, and a sliding window of size 4 is made for the weld feature point queue acquired in real time by the line structured light, such as Figure 9 As shown in the figure, the characteristic points in the sliding window are extracted, and the control points required for the improved B-spline are added to the two middle points. The improved B-spline fitting is performed on the six points. The fitting curve should just pass through the two characteristic points. The fitting curve is uniformly discretized. If there is a starting point or an ending point in the sliding window, two duplicate points are added to the starting point or the ending point, and the improved B-spline fitting is performed on the nine points, and uniform discretization is performed to ensure that the curve passes through the starting point or the ending point. The path point information is then fed back to the robot, allowing it to perform the welding task according to the fitted path.

[0125] The processing of abnormal points is as follows: set a sliding window with a size of N. The value of N needs to take into account the welding speed, tracking accuracy and real-time performance. Calculate the difference between two adjacent feature points in the Y and Z directions, recorded as Y i and Z i , the difference between the point to be determined and its previous feature point is recorded as Y new and Z new , the Z-score method is used to determine whether the difference is an outlier, and the mean and variance of the difference are calculated:

[0126]

[0127]

[0128]

[0129]

[0130] Then calculate the Z-score of the difference between the discriminant points:

[0131]

[0132]

[0133] When | Z new,y |>T or |Z new,z When |>T, the discriminant point can be identified as an abnormal point.

[0134] Prior to this, the specific implementation principle of step S4 is as follows: since the problem of invalid feature point recognition occurs during welding, it is necessary to distinguish the invalid points appearing in the middle from the weld end point, and thus a threshold range is set according to the taught weld end point information. When the identified weld feature point reaches this threshold range, when two or more invalid points appear consecutively, the last point is determined to be the weld end point.

[0135] like Figure 2 As shown, another object of the present invention is to provide a real-time weld tracking system based on 3D point cloud, comprising:

[0136] The recognition module is used to select points on the weld to be welded, obtain point cloud data of the points through line structured light, identify weld feature points, and complete the coordinate conversion between the weld feature points and the robot welding end through hand-eye calibration;

[0137] The tracking module is used to teach an initial path for seam tracking. When the robot identifies a weld feature point along the path, this point is defined as the weld starting point. The robot moves to the weld starting point along the welding speed and waits for the welding instruction before starting seam tracking.

[0138] The fitting module is used to perform improved B-spline fitting on the weld feature points in the weld feature point queue when the robot terminal reaches the weld starting point due to the line structured light advance, perform uniform discretization, and generate smooth and accurate discrete points as the robot's motion path;

[0139] The discrimination module is used to discriminate the end point of the weld. In the weld feature point queue, it takes into account the influence of recognition error factors and discriminates the invalid points and end points that appear in the welding to complete the entire welding task.

[0140] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the real-time weld tracking method based on 3D point cloud.

[0141] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the real-time weld tracking method based on 3D point cloud.

[0142] Another object of the present invention is to provide an information data processing terminal, which is used to implement the real-time weld tracking system based on 3D point cloud.

[0143] The present invention is specifically implemented:

[0144] The present invention provides a weld seam tracking method based on 3D point cloud. The whole weld seam tracking method steps are as follows: Figure 4 As shown, the hardware of the entire weld tracking system includes a robot 1, a structured light sensor 2, a computer 3, an industrial computer 4, and a welding machine 5. Figure 3 As shown;

[0145] The workpiece to be welded is placed under the welding torch. The structured light sensor is placed at the front of the torch and connected to a computer, which reads and processes the point cloud information transmitted by the structured light sensor. The computer is also connected to the robot's industrial computer for real-time communication. The industrial computer controls the robot's movement and the welding parameters of the welding machine.

[0146] Step S1 uses a line structured light camera to obtain weld point cloud information, and a computer processes the point cloud data to obtain weld feature point information. The specific principle is as follows:

[0147] In step S1-1, select the weld type to be identified on the computer. Here, we'll use butt welds as an example. Using a straight-through filter, we set upper and lower limits. Due to the characteristics of butt welds, we'll set the z range to (-0.5, 2) and the y range to (-4, 4), limiting the weld point cloud data to a region of interest. This approach significantly reduces the amount of point cloud data, accelerating subsequent processing steps.

[0148] Step S1-2 divides the point cloud data into several small three-dimensional grid units (voxels) through voxel grid filtering, and then replaces all points in each voxel with a representative point (such as the centroid) of the points in each voxel, thereby achieving the effect of data downsampling.

[0149] In step S1-3, the point cloud is segmented into different clusters according to the point cloud density through the DBSCAN method. Each cluster corresponds to an independent geometric structure, and the two planes of the butt weld are segmented. The distance between each point is calculated by the difference method. According to the set threshold of 0.5, the weld area of ​​the butt weld can be identified, and the boundary points of the two segmented surfaces are obtained. The center point of the two boundary points is calculated as the weld feature point.

[0150] Step S2 calibrates the weld feature points and welding gun using the 10-point method. First, a weld point is selected on the weld. According to the welding parameters required for welding, such as the focal length and incident angle of laser welding, the end effector is placed on the weld point and the weld feature points identified by the camera at different positions at this point are recorded. The transformation matrix of the weld feature points identified by the end effector and camera is calculated according to the principle of the 10-point method and recorded.

[0151] Step S3 manually teaches an initial trajectory. It is necessary to ensure that the camera can recognize the weld feature points as the robot moves along the trajectory. The end point is near the end of the weld. The robot is allowed to move along the taught path. When the camera recognizes the weld feature points and two or more feature points appear continuously in the feature point queue, the robot is allowed to run to the weld starting point at the welding speed.

[0152] In step S4, due to the advance of line structured light, there are K feature point values ​​in the weld feature point column, where K is the quotient of the advance distance divided by the period of line structured light acquisition of point cloud data. Due to the continuity of the B-spline curve, the feature points can be segmented, and the number of segmented feature points is α*K, where α depends on the welding speed and the period of acquisition of point cloud. The fitted improved B-spline is uniformly discretized, and the discrete number is m*α*K, where m is the quotient of the period of acquisition of point cloud and the period of robot motion. The discretized point information is transmitted to the robot industrial computer, so that the robot performs welding along the path.

[0153] Step S5 is for determining the end point of the weld. Since the problem of invalid feature point recognition occurs during welding, it is necessary to distinguish the invalid points that appear in the middle from the weld end point. Therefore, a threshold range is set according to the point position information of the taught weld end point. When the identified weld feature point reaches this threshold range, when two or more invalid points appear continuously, the last point is determined to be the weld end point.

[0154] The processing of weld abnormal points is as follows: set a sliding window with a size of N. The value of N needs to take into account the welding speed, tracking accuracy and real-time performance. Calculate the difference between two adjacent feature points in the Y and Z directions, recorded as Y i and Z i , the difference between the point to be determined and its previous feature point is recorded as Y new and Z new , the Z-score method is used to determine whether the difference is an outlier, and the mean and variance of the difference are calculated:

[0155]

[0156]

[0157]

[0158]

[0159] Then calculate the Z-score of the difference between the discriminant points:

[0160]

[0161]

[0162] When | Z new，y |>3 or |Z new，z When |>3, the discriminant point can be identified as an abnormal point.

[0163] After determining the abnormal point, a point will be fitted to replace the abnormal point based on the mean and variance calculated in the sliding window to avoid the problem of welding speed change caused by the increase in point distance due to the lack of points in the subsequent uniform discretization.

[0164] Step S6: After the welding task is completed, the robot returns to a safe position.

[0165] Application Example 1: Industrial Robot Welding System

[0166] Welding is a critical process in industrial manufacturing. Traditional welding operations typically rely on manual labor or simple automated systems. These methods are susceptible to the effects of worker skill and the welding environment, resulting in unstable weld quality. This paper proposes a welding system based on robotics and intelligent algorithms that can achieve high-quality automated welding in complex welding environments.

[0167] 1) Hand-eye calibration:

[0168] The 10-point method is used to perform hand-eye calibration in different positions and postures.

[0169] The position of the robot end effector and the corresponding feature point positions are collected to obtain the pose matrix.

[0170] The fixed transformation matrix is ​​solved by SVD and least squares method to ensure high-precision calibration.

[0171] 2) Path teaching:

[0172] Manually teach the initial welding path and set welding parameters to ensure that the structured light can identify the weld.

[0173] According to statistical principles, when two or more characteristic points appear consecutively, the first characteristic point is taken as the starting point of the weld.

[0174] 3) Path fitting:

[0175] The improved B-spline curve is used to fit the weld characteristic points, and the fitting curve is evenly discretized.

[0176] The path point information is fed back to the robot so that it can perform welding tasks according to the fitted path.

[0177] Set a sliding window and use the Z-score method to handle outliers to ensure path smoothness and accuracy.

[0178] 4) End point determination:

[0179] Set the threshold range based on the taught weld end point information.

[0180] When the identified weld feature points reach the threshold range and two or more invalid points appear continuously, the weld end point is determined.

[0181] Through the above steps, the present invention realizes high-precision hand-eye calibration, efficient path teaching and fitting, and accurate end point judgment in industrial robot welding, which significantly improves welding quality and efficiency.

[0182] Application Example 2: Bridge Steel Structure Welding System

[0183] The welding of bridge steel structures requires high precision and high reliability. Traditional welding methods are unable to meet the welding requirements of complex structures. The intelligent welding system proposed in this paper can achieve high-quality automated welding in complex bridge steel structures, ensuring the structural strength and durability of the bridge.

[0184] 1) Hand-eye calibration:

[0185] The 10-point method is used for hand-eye calibration to collect the pose matrix of the robot end effector and camera.

[0186] The fixed transformation matrix is ​​solved by SVD and least squares method to ensure high-precision calibration.

[0187] 2) Path teaching:

[0188] Manually teach the initial welding path and set welding parameters to ensure that the structured light can identify the weld.

[0189] According to statistical principles, when two or more characteristic points appear consecutively, the first characteristic point is taken as the starting point of the weld.

[0190] 3) Path fitting:

[0191] The improved B-spline curve is used to fit the weld characteristic points, and the fitting curve is evenly discretized.

[0192] The path point information is fed back to the robot so that it can perform welding tasks according to the fitted path.

[0193] Set a sliding window and use the Z-score method to handle outliers to ensure path smoothness and accuracy.

[0194] 4) End point determination:

[0195] Set the threshold range based on the taught weld end point information.

[0196] When the identified weld feature points reach the threshold range and two or more invalid points appear continuously, the weld end point is determined.

[0197] In the welding of bridge steel structures, the present invention ensures high quality and high reliability of welding through high-precision hand-eye calibration, intelligent path teaching and fitting, and accurate endpoint judgment, providing strong protection for the safety and durability of bridge steel structures.

[0198] These two embodiments demonstrate the specific operating steps and significant technical progress of the present invention in different application scenarios, effectively solving multiple key problems in the prior art.

[0199] The relevant evidence of the technical effects achieved by the embodiments of the present invention is as follows:

[0200] The present invention simulates the sampling characteristics of the structured light camera and uses MATLAB to sample a 1 / 6 arc with a radius of 400 mm. The simulated welding speed is 50 mm / s, the sampling period is 40 ms, the robot motion period is 4 ms, and a sampling point is taken every 2 mm. Figure 5 As shown, for clarity, every 10 points are shown as a feature point and recorded in the feature point queue.

[0201] First, perform the above-mentioned improved B-spline interpolation directly on the feature points, and bring the obtained x-coordinates of the discrete points into the arc curve, calculate the error between the discrete points and the corresponding y-coordinates of the actual arc points, and regard it as the theoretical error of the curve. Figure 6 As shown in the figure, the theoretical error between the discrete points on the curve and the actual arc is within ±1×10^-9.

[0202] Taking into account the recognition error and manual hand-eye calibration error of the sampling points in the actual welding process, the points of the arc curve are randomly fluctuated by ±0.2mm, and the above method is used to fit the discretization. The error between the discrete points and the actual points of the arc curve under the same x-coordinate is calculated, and the speed error is also calculated. Figure 7 It can be seen that under the condition of 0.2mm feature point error, the maximum actual error of the fitting position is 0.2181mm, and the calculated average error is 0.09mm. Figure 8 As can be seen, the actual motion speed has a maximum speed error of 0.6371 mm / s compared to the theoretical speed of 50 mm / s, and the calculated average speed error is 0.082 mm / s, which meets welding requirements. The entire arc trajectory fitting time is 117 ms. Because motion and trajectory fitting are performed simultaneously during real-time welding, and curve fitting precedes robot motion, the real-time performance also meets welding requirements.

[0203] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art will appreciate that the above-mentioned devices and methods can be implemented using computer-executable instructions and / or contained in processor control code, for example, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, or can be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0204] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A real-time weld tracking method based on 3D point cloud, characterized in that: The following steps are involved: S1. Select points on the weld to be welded, obtain point cloud data of the points through line structured light, identify weld feature points, and complete the coordinate conversion between the weld feature points and the robot welding end through hand-eye calibration; S2. Teach an initial path for seam tracking. When the robot recognizes a weld feature point along the path, this point is defined as the weld starting point. The robot moves to the weld starting point along the welding speed and waits for the welding instruction before starting seam tracking. S3. Due to the line structured light in advance, when the end of the robot reaches the starting point of the weld, the weld feature points in the weld feature point queue are uniformly discretized by improved B-spline fitting to generate smooth and accurate discrete points as the robot's motion path. S4. Weld end point determination: In the weld feature point queue, considering the influence of recognition error factors, invalid points and end points that appear in the welding process are determined to complete the entire welding task. The step S3 method: The feature points in the weld feature point queue are fitted with an improved B-spline curve, and the fitted curve is uniformly discretized. The path point information is fed back to the robot so that it can perform the welding task according to the fitted path. The processing of abnormal points is as follows: set a sliding window with a size of N. The value of N needs to take into account the welding speed, tracking accuracy and real-time performance; calculate the difference between two adjacent feature points in the Y and Z directions, which is recorded as and , the difference between the point to be determined and its previous feature point is recorded as and , the Z-score method is used to determine whether the difference is an outlier, and the mean and variance of the difference are calculated: ； ； ； ； Then calculate the Z-score of the difference between the discriminant points: ； ； when or , the discriminant point can be identified as an abnormal point.

2. The real-time weld seam tracking method based on 3D point cloud according to claim 1, characterized in that: The method for identifying weld feature points using a line structured light camera: First, the point cloud is preprocessed by removing some weld points outside the weld through straight-through filtering to make the point cloud data more concentrated in the weld area. Then, the point cloud density is reduced through voxel grid filtering. The weld surface is classified through DBSCAN. For weld feature point identification, different solutions are adopted for different welds, as shown below: For butt or V-groove welds, the classified weld surfaces are identified using the differential method, and the center point of the weld boundary point is used as the weld feature point; For lap welds, the classified weld surfaces are used to identify the welds using the differential method, and the higher weld surface boundary is used as the weld feature point; For fillet welds, a straight line is fitted to the classified weld surface, and the intersection of the two straight lines is the weld feature point.

3. The real-time weld seam tracking method based on 3D point cloud according to claim 1, characterized in that: The specific method of step S1 is as follows: The 10-point method is used for hand-eye calibration. First, the position of the robot end effector and the position of the identified feature points are collected at different positions and postures. At each position and posture, the pose matrix of the robot end effector can be obtained. and the camera pose matrix , we can construct the equation: ； ； ； Where X is a fixed transformation matrix, is the rotation matrix, and are the translation vectors of the robot end effector and the camera at the i-th position and posture respectively, converted into vector form. is the translation vector; SVD can be used to solve it. Use the least squares method to solve.

4. The real-time weld seam tracking method based on 3D point cloud according to claim 1, characterized in that: The step S2 method: Manually teach an initial path and set the required welding parameters. It is necessary to ensure that the structured light can recognize the weld. When identifying the weld feature points, according to statistical principles, when two or more feature points appear consecutively, the first feature point is used as the starting point of the weld.

5. The real-time weld seam tracking method based on 3D point cloud according to claim 1, characterized in that: The step S4 method: A threshold range is set according to the taught weld end point information. When the identified weld feature point reaches this threshold range, when two or more invalid points appear consecutively, the last point is determined to be the weld end point.

6. A 3D point cloud-based real-time weld seam tracking system implementing the 3D point cloud-based real-time weld seam tracking method according to any one of claims 1 to 5, characterized in that: The real-time weld seam tracking system based on 3D point cloud includes: The recognition module is used to select points on the weld to be welded, obtain point cloud data of the points through line structured light, identify weld feature points, and complete the coordinate conversion between the weld feature points and the robot welding end through hand-eye calibration; The tracking module is used to teach an initial path for seam tracking. When the robot identifies a weld feature point along the path, this point is defined as the weld starting point. The robot moves to the weld starting point along the welding speed and waits for the welding instruction before starting seam tracking. The fitting module is used to perform improved B-spline fitting on the weld feature points in the weld feature point queue when the robot terminal reaches the weld starting point due to the line structured light advance, perform uniform discretization, and generate smooth and accurate discrete points as the robot's motion path; The discrimination module is used to discriminate the end point of the weld. In the weld feature point queue, it takes into account the influence of recognition error factors and discriminates the invalid points and end points that appear in the welding to complete the entire welding task.

7. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the real-time weld tracking method based on 3D point cloud as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the real-time weld tracking method based on 3D point cloud according to any one of claims 1 to 5.

9. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the real-time weld tracking system based on 3D point cloud as described in claim 6.

Images